agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feed[PATCH v16] Avoid orphaned objects dependencies
331+ messages / 4 participants
[nested] [flat]
* [PATCH v16] Avoid orphaned objects dependencies
@ 2024-03-29 15:43 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2024-03-29 15:43 UTC (permalink / raw)
It's currently possible to create orphaned objects dependencies, for example:
Scenario 1:
session 1: begin; drop schema schem;
session 2: create a function in the schema schem
session 1: commit;
With the above, the function created in session 2 would be linked to a non
existing schema.
Scenario 2:
session 1: begin; create a function in the schema schem
session 2: drop schema schem;
session 1: commit;
With the above, the function created in session 1 would be linked to a non
existing schema.
To avoid those scenarios, a new lock (that conflicts with a lock taken by DROP)
has been put in place before the dependencies are being recorded. With this in
place, the drop schema in scenario 2 would be locked.
Also, after the new lock attempt, the patch checks that the object still exists:
with this in place session 2 in scenario 1 would be locked and would report an
error once session 1 committs (that would not be the case should session 1 abort
the transaction).
If the object is dropped before the new lock attempt is triggered then the patch
would also report an error (but with less details).
The patch takes into account any type of objects except the ones that are pinned
(they are not droppable because the system requires it).
A special case is done for objects that belong to the RelationRelationId class.
For those, we should be in one of the two following cases that would already
prevent the relation to be dropped:
1. The relation is already locked (could be an existing relation or a relation
that we are creating).
2. The relation is protected indirectly (i.e an index protected by a lock on
its table, a table protected by a lock on a function that depends the table...)
To avoid any risks for the RelationRelationId class case, we acquire a lock if
there is none. That may add unnecessary lock for 2. but that's worth it.
The patch adds a few tests for some dependency cases (that would currently produce
orphaned objects):
- schema and function (as the above scenarios)
- alter a dependency (function and schema)
- function and arg type
- function and return type
- function and function
- domain and domain
- table and type
- server and foreign data wrapper
---
src/backend/catalog/aclchk.c | 1 +
src/backend/catalog/dependency.c | 212 ++++++++++++++++++
src/backend/catalog/heap.c | 7 +
src/backend/catalog/index.c | 26 +++
src/backend/catalog/objectaddress.c | 57 +++++
src/backend/catalog/pg_aggregate.c | 9 +
src/backend/catalog/pg_attrdef.c | 1 +
src/backend/catalog/pg_cast.c | 5 +
src/backend/catalog/pg_collation.c | 1 +
src/backend/catalog/pg_constraint.c | 26 +++
src/backend/catalog/pg_conversion.c | 2 +
src/backend/catalog/pg_depend.c | 40 +++-
src/backend/catalog/pg_operator.c | 19 ++
src/backend/catalog/pg_proc.c | 17 +-
src/backend/catalog/pg_publication.c | 11 +
src/backend/catalog/pg_range.c | 6 +
src/backend/catalog/pg_type.c | 39 ++++
src/backend/catalog/toasting.c | 1 +
src/backend/commands/alter.c | 4 +
src/backend/commands/amcmds.c | 1 +
src/backend/commands/cluster.c | 7 +
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/extension.c | 5 +
src/backend/commands/foreigncmds.c | 7 +
src/backend/commands/functioncmds.c | 6 +
src/backend/commands/indexcmds.c | 2 +
src/backend/commands/opclasscmds.c | 18 ++
src/backend/commands/operatorcmds.c | 30 +++
src/backend/commands/policy.c | 2 +
src/backend/commands/proclang.c | 3 +
src/backend/commands/sequence.c | 2 +
src/backend/commands/statscmds.c | 10 +
src/backend/commands/tablecmds.c | 34 ++-
src/backend/commands/trigger.c | 29 ++-
src/backend/commands/tsearchcmds.c | 73 +++++-
src/backend/commands/typecmds.c | 84 +++++++
src/backend/rewrite/rewriteDefine.c | 1 +
src/backend/utils/errcodes.txt | 1 +
src/include/catalog/dependency.h | 3 +
src/include/catalog/objectaddress.h | 1 +
.../expected/test_dependencies_locks.out | 129 +++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/test_dependencies_locks.spec | 89 ++++++++
.../test_oat_hooks/expected/alter_table.out | 4 +-
.../expected/test_oat_hooks.out | 2 +
src/test/regress/expected/alter_table.out | 11 +-
46 files changed, 1015 insertions(+), 25 deletions(-)
39.7% src/backend/catalog/
30.4% src/backend/commands/
16.7% src/test/isolation/expected/
10.4% src/test/isolation/specs/
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index a44ccee3b6..9a24872a30 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -1413,6 +1413,7 @@ SetDefaultACL(InternalDefaultACL *iacls)
referenced.objectId = iacls->nspid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, iacls->nspid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
}
}
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0489cbabcb..a3770d7206 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1519,6 +1519,81 @@ AcquireDeletionLock(const ObjectAddress *object, int flags)
}
}
+/*
+ * LockNotPinnedObjectById
+ *
+ * Lock the object that we are about to record a dependency on.
+ * After it's locked, verify that it hasn't been dropped while we
+ * weren't looking. If the object has been dropped, this function
+ * does not return!
+ */
+void
+LockNotPinnedObjectById(const ObjectAddress *object)
+{
+ char *object_description = NULL;
+
+ if (isObjectPinned(object))
+ return;
+
+ object_description = getObjectDescription(object, true);
+
+ if (object->classId == RelationRelationId)
+ {
+ Assert(!IsSharedRelation(object->objectId));
+
+ /*
+ * We must be in one of the two following cases that would already
+ * prevent the relation to be dropped: 1. The relation is already
+ * locked (could be an existing relation or a relation that we are
+ * creating). 2. The relation is protected indirectly (i.e an index
+ * protected by a lock on its table, a table protected by a lock on a
+ * function that depends of the table...). To avoid any risks, acquire
+ * a lock if there is none. That may add unnecessary lock for 2. but
+ * that's worth it.
+ */
+ if (!CheckRelationOidLockedByMe(object->objectId, AccessShareLock, true))
+ LockRelationOid(object->objectId, AccessShareLock);
+ }
+ else
+ {
+ /* assume we should lock the whole object not a sub-object */
+ LockDatabaseObject(object->classId, object->objectId, 0, AccessShareLock);
+ }
+
+ /* check if object still exists */
+ if (!ObjectByIdExist(object))
+ {
+ if (object_description)
+ ereport(ERROR,
+ (errcode(ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST),
+ errmsg("%s does not exist", object_description)));
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST),
+ errmsg("a dependent object does not exist")));
+ }
+
+ if (object_description)
+ pfree(object_description);
+
+ return;
+}
+
+/*
+ * LockNotPinnedObject
+ *
+ * Lock the object that we are about to record a dependency on.
+ */
+void
+LockNotPinnedObject(Oid classid, Oid objid)
+{
+ ObjectAddress object;
+
+ ObjectAddressSet(object, classid, objid);
+
+ LockNotPinnedObjectById(&object);
+}
+
/*
* ReleaseDeletionLock - release an object deletion lock
*
@@ -1730,6 +1805,7 @@ find_expr_references_walker(Node *node,
if (rte->rtekind == RTE_RELATION)
{
/* If it's a plain relation, reference this column */
+ LockNotPinnedObject(RelationRelationId, rte->relid);
add_object_address(RelationRelationId, rte->relid, var->varattno,
context->addrs);
}
@@ -1756,6 +1832,7 @@ find_expr_references_walker(Node *node,
Oid objoid;
/* A constant must depend on the constant's datatype */
+ LockNotPinnedObject(TypeRelationId, con->consttype);
add_object_address(TypeRelationId, con->consttype, 0,
context->addrs);
@@ -1767,8 +1844,11 @@ find_expr_references_walker(Node *node,
*/
if (OidIsValid(con->constcollid) &&
con->constcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, con->constcollid);
add_object_address(CollationRelationId, con->constcollid, 0,
context->addrs);
+ }
/*
* If it's a regclass or similar literal referring to an existing
@@ -1785,59 +1865,83 @@ find_expr_references_walker(Node *node,
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(PROCOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(ProcedureRelationId, objoid);
add_object_address(ProcedureRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGOPEROID:
case REGOPERATOROID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(OPEROID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(OperatorRelationId, objoid);
add_object_address(OperatorRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCLASSOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(RELOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(RelationRelationId, objoid);
add_object_address(RelationRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGTYPEOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TYPEOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TypeRelationId, objoid);
add_object_address(TypeRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCOLLATIONOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(COLLOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(CollationRelationId, objoid);
add_object_address(CollationRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCONFIGOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TSCONFIGOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TSConfigRelationId, objoid);
add_object_address(TSConfigRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGDICTIONARYOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TSDICTOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TSDictionaryRelationId, objoid);
add_object_address(TSDictionaryRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGNAMESPACEOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(NAMESPACEOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(NamespaceRelationId, objoid);
add_object_address(NamespaceRelationId, objoid, 0,
context->addrs);
+ }
break;
/*
@@ -1859,18 +1963,23 @@ find_expr_references_walker(Node *node,
Param *param = (Param *) node;
/* A parameter must depend on the parameter's datatype */
+ LockNotPinnedObject(TypeRelationId, param->paramtype);
add_object_address(TypeRelationId, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
param->paramcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, param->paramcollid);
add_object_address(CollationRelationId, param->paramcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, FuncExpr))
{
FuncExpr *funcexpr = (FuncExpr *) node;
+ LockNotPinnedObject(ProcedureRelationId, funcexpr->funcid);
add_object_address(ProcedureRelationId, funcexpr->funcid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1879,6 +1988,7 @@ find_expr_references_walker(Node *node,
{
OpExpr *opexpr = (OpExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, opexpr->opno);
add_object_address(OperatorRelationId, opexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1887,6 +1997,7 @@ find_expr_references_walker(Node *node,
{
DistinctExpr *distinctexpr = (DistinctExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, distinctexpr->opno);
add_object_address(OperatorRelationId, distinctexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1895,6 +2006,7 @@ find_expr_references_walker(Node *node,
{
NullIfExpr *nullifexpr = (NullIfExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, nullifexpr->opno);
add_object_address(OperatorRelationId, nullifexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1903,6 +2015,7 @@ find_expr_references_walker(Node *node,
{
ScalarArrayOpExpr *opexpr = (ScalarArrayOpExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, opexpr->opno);
add_object_address(OperatorRelationId, opexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1911,6 +2024,7 @@ find_expr_references_walker(Node *node,
{
Aggref *aggref = (Aggref *) node;
+ LockNotPinnedObject(ProcedureRelationId, aggref->aggfnoid);
add_object_address(ProcedureRelationId, aggref->aggfnoid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1919,6 +2033,7 @@ find_expr_references_walker(Node *node,
{
WindowFunc *wfunc = (WindowFunc *) node;
+ LockNotPinnedObject(ProcedureRelationId, wfunc->winfnoid);
add_object_address(ProcedureRelationId, wfunc->winfnoid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1935,8 +2050,11 @@ find_expr_references_walker(Node *node,
*/
if (sbsref->refrestype != sbsref->refcontainertype &&
sbsref->refrestype != sbsref->refelemtype)
+ {
+ LockNotPinnedObject(TypeRelationId, sbsref->refrestype);
add_object_address(TypeRelationId, sbsref->refrestype, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, SubPlan))
@@ -1960,16 +2078,25 @@ find_expr_references_walker(Node *node,
* anywhere else in the expression.
*/
if (OidIsValid(reltype))
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
add_object_address(RelationRelationId, reltype, fselect->fieldnum,
context->addrs);
+ }
else
+ {
+ LockNotPinnedObject(TypeRelationId, fselect->resulttype);
add_object_address(TypeRelationId, fselect->resulttype, 0,
context->addrs);
+ }
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
fselect->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, fselect->resultcollid);
add_object_address(CollationRelationId, fselect->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, FieldStore))
{
@@ -1980,53 +2107,76 @@ find_expr_references_walker(Node *node,
if (OidIsValid(reltype))
{
ListCell *l;
+ bool locked = false;
foreach(l, fstore->fieldnums)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
+ locked = true;
+ }
add_object_address(RelationRelationId, reltype, lfirst_int(l),
context->addrs);
+ }
}
else
+ {
+ LockNotPinnedObject(TypeRelationId, fstore->resulttype);
add_object_address(TypeRelationId, fstore->resulttype, 0,
context->addrs);
+ }
}
else if (IsA(node, RelabelType))
{
RelabelType *relab = (RelabelType *) node;
/* since there is no function dependency, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, relab->resulttype);
add_object_address(TypeRelationId, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
relab->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, relab->resultcollid);
add_object_address(CollationRelationId, relab->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, CoerceViaIO))
{
CoerceViaIO *iocoerce = (CoerceViaIO *) node;
/* since there is no exposed function, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, iocoerce->resulttype);
add_object_address(TypeRelationId, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, iocoerce->resultcollid);
add_object_address(CollationRelationId, iocoerce->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, ArrayCoerceExpr))
{
ArrayCoerceExpr *acoerce = (ArrayCoerceExpr *) node;
/* as above, depend on type */
+ LockNotPinnedObject(TypeRelationId, acoerce->resulttype);
add_object_address(TypeRelationId, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, acoerce->resultcollid);
add_object_address(CollationRelationId, acoerce->resultcollid, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, ConvertRowtypeExpr))
@@ -2034,6 +2184,7 @@ find_expr_references_walker(Node *node,
ConvertRowtypeExpr *cvt = (ConvertRowtypeExpr *) node;
/* since there is no function dependency, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, cvt->resulttype);
add_object_address(TypeRelationId, cvt->resulttype, 0,
context->addrs);
}
@@ -2041,6 +2192,7 @@ find_expr_references_walker(Node *node,
{
CollateExpr *coll = (CollateExpr *) node;
+ LockNotPinnedObject(CollationRelationId, coll->collOid);
add_object_address(CollationRelationId, coll->collOid, 0,
context->addrs);
}
@@ -2048,6 +2200,7 @@ find_expr_references_walker(Node *node,
{
RowExpr *rowexpr = (RowExpr *) node;
+ LockNotPinnedObject(TypeRelationId, rowexpr->row_typeid);
add_object_address(TypeRelationId, rowexpr->row_typeid, 0,
context->addrs);
}
@@ -2058,11 +2211,13 @@ find_expr_references_walker(Node *node,
foreach(l, rcexpr->opnos)
{
+ LockNotPinnedObject(OperatorRelationId, lfirst_oid(l));
add_object_address(OperatorRelationId, lfirst_oid(l), 0,
context->addrs);
}
foreach(l, rcexpr->opfamilies)
{
+ LockNotPinnedObject(OperatorFamilyRelationId, lfirst_oid(l));
add_object_address(OperatorFamilyRelationId, lfirst_oid(l), 0,
context->addrs);
}
@@ -2072,6 +2227,7 @@ find_expr_references_walker(Node *node,
{
CoerceToDomain *cd = (CoerceToDomain *) node;
+ LockNotPinnedObject(TypeRelationId, cd->resulttype);
add_object_address(TypeRelationId, cd->resulttype, 0,
context->addrs);
}
@@ -2079,6 +2235,7 @@ find_expr_references_walker(Node *node,
{
NextValueExpr *nve = (NextValueExpr *) node;
+ LockNotPinnedObject(RelationRelationId, nve->seqid);
add_object_address(RelationRelationId, nve->seqid, 0,
context->addrs);
}
@@ -2087,19 +2244,26 @@ find_expr_references_walker(Node *node,
OnConflictExpr *onconflict = (OnConflictExpr *) node;
if (OidIsValid(onconflict->constraint))
+ {
+ LockNotPinnedObject(ConstraintRelationId, onconflict->constraint);
add_object_address(ConstraintRelationId, onconflict->constraint, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, SortGroupClause))
{
SortGroupClause *sgc = (SortGroupClause *) node;
+ LockNotPinnedObject(OperatorRelationId, sgc->eqop);
add_object_address(OperatorRelationId, sgc->eqop, 0,
context->addrs);
if (OidIsValid(sgc->sortop))
+ {
+ LockNotPinnedObject(OperatorRelationId, sgc->sortop);
add_object_address(OperatorRelationId, sgc->sortop, 0,
context->addrs);
+ }
return false;
}
else if (IsA(node, WindowClause))
@@ -2107,15 +2271,24 @@ find_expr_references_walker(Node *node,
WindowClause *wc = (WindowClause *) node;
if (OidIsValid(wc->startInRangeFunc))
+ {
+ LockNotPinnedObject(ProcedureRelationId, wc->startInRangeFunc);
add_object_address(ProcedureRelationId, wc->startInRangeFunc, 0,
context->addrs);
+ }
if (OidIsValid(wc->endInRangeFunc))
+ {
+ LockNotPinnedObject(ProcedureRelationId, wc->endInRangeFunc);
add_object_address(ProcedureRelationId, wc->endInRangeFunc, 0,
context->addrs);
+ }
if (OidIsValid(wc->inRangeColl) &&
wc->inRangeColl != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, wc->inRangeColl);
add_object_address(CollationRelationId, wc->inRangeColl, 0,
context->addrs);
+ }
/* fall through to examine substructure */
}
else if (IsA(node, CTECycleClause))
@@ -2123,14 +2296,23 @@ find_expr_references_walker(Node *node,
CTECycleClause *cc = (CTECycleClause *) node;
if (OidIsValid(cc->cycle_mark_type))
+ {
+ LockNotPinnedObject(TypeRelationId, cc->cycle_mark_type);
add_object_address(TypeRelationId, cc->cycle_mark_type, 0,
context->addrs);
+ }
if (OidIsValid(cc->cycle_mark_collation))
+ {
+ LockNotPinnedObject(CollationRelationId, cc->cycle_mark_collation);
add_object_address(CollationRelationId, cc->cycle_mark_collation, 0,
context->addrs);
+ }
if (OidIsValid(cc->cycle_mark_neop))
+ {
+ LockNotPinnedObject(OperatorRelationId, cc->cycle_mark_neop);
add_object_address(OperatorRelationId, cc->cycle_mark_neop, 0,
context->addrs);
+ }
/* fall through to examine substructure */
}
else if (IsA(node, Query))
@@ -2163,6 +2345,7 @@ find_expr_references_walker(Node *node,
switch (rte->rtekind)
{
case RTE_RELATION:
+ LockNotPinnedObject(RelationRelationId, rte->relid);
add_object_address(RelationRelationId, rte->relid, 0,
context->addrs);
break;
@@ -2215,12 +2398,18 @@ find_expr_references_walker(Node *node,
rte = rt_fetch(query->resultRelation, query->rtable);
if (rte->rtekind == RTE_RELATION)
{
+ bool locked = false;
foreach(lc, query->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
if (tle->resjunk)
continue; /* ignore junk tlist items */
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, rte->relid);
+ locked = true;
+ }
add_object_address(RelationRelationId, rte->relid, tle->resno,
context->addrs);
}
@@ -2232,6 +2421,7 @@ find_expr_references_walker(Node *node,
*/
foreach(lc, query->constraintDeps)
{
+ LockNotPinnedObject(ConstraintRelationId, lfirst_oid(lc));
add_object_address(ConstraintRelationId, lfirst_oid(lc), 0,
context->addrs);
}
@@ -2266,16 +2456,25 @@ find_expr_references_walker(Node *node,
*/
foreach(ct, rtfunc->funccoltypes)
{
+ LockNotPinnedObject(TypeRelationId, lfirst_oid(ct));
add_object_address(TypeRelationId, lfirst_oid(ct), 0,
context->addrs);
}
foreach(ct, rtfunc->funccolcollations)
{
Oid collid = lfirst_oid(ct);
+ bool locked = false;
if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(CollationRelationId, collid);
+ locked = true;
+ }
add_object_address(CollationRelationId, collid, 0,
context->addrs);
+ }
}
}
else if (IsA(node, TableFunc))
@@ -2288,22 +2487,32 @@ find_expr_references_walker(Node *node,
*/
foreach(ct, tf->coltypes)
{
+ LockNotPinnedObject(TypeRelationId, lfirst_oid(ct));
add_object_address(TypeRelationId, lfirst_oid(ct), 0,
context->addrs);
}
foreach(ct, tf->colcollations)
{
Oid collid = lfirst_oid(ct);
+ bool locked = false;
if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(CollationRelationId, collid);
+ locked = true;
+ }
add_object_address(CollationRelationId, collid, 0,
context->addrs);
+ }
}
}
else if (IsA(node, TableSampleClause))
{
TableSampleClause *tsc = (TableSampleClause *) node;
+ LockNotPinnedObject(ProcedureRelationId, tsc->tsmhandler);
add_object_address(ProcedureRelationId, tsc->tsmhandler, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2354,9 +2563,12 @@ process_function_rte_ref(RangeTblEntry *rte, AttrNumber attnum,
Assert(attnum - atts_done <= tupdesc->natts);
if (OidIsValid(reltype)) /* can this fail? */
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
add_object_address(RelationRelationId, reltype,
attnum - atts_done,
context->addrs);
+ }
return;
}
/* Nothing to do; function's result type is handled elsewhere */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 01b43cc6a8..97600e7a3b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -844,6 +844,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* Add dependency info */
ObjectAddressSubSet(myself, RelationRelationId, new_rel_oid, i + 1);
ObjectAddressSet(referenced, TypeRelationId, attr->atttypid);
+ LockNotPinnedObject(TypeRelationId, attr->atttypid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* The default collation is pinned, so don't bother recording it */
@@ -852,6 +853,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
{
ObjectAddressSet(referenced, CollationRelationId,
attr->attcollation);
+ LockNotPinnedObject(CollationRelationId, attr->attcollation);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -1459,11 +1461,13 @@ heap_create_with_catalog(const char *relname,
ObjectAddressSet(referenced, NamespaceRelationId, relnamespace);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(NamespaceRelationId, relnamespace);
if (reloftypeid)
{
ObjectAddressSet(referenced, TypeRelationId, reloftypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, reloftypeid);
}
/*
@@ -1477,6 +1481,7 @@ heap_create_with_catalog(const char *relname,
{
ObjectAddressSet(referenced, AccessMethodRelationId, accessmtd);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(AccessMethodRelationId, accessmtd);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -3391,6 +3396,7 @@ StorePartitionKey(Relation rel,
{
ObjectAddressSet(referenced, OperatorClassRelationId, partopclass[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, partopclass[i]);
/* The default collation is pinned, so don't bother recording it */
if (OidIsValid(partcollation[i]) &&
@@ -3398,6 +3404,7 @@ StorePartitionKey(Relation rel,
{
ObjectAddressSet(referenced, CollationRelationId, partcollation[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, partcollation[i]);
}
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a819b4197c..d6d1abfcf5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1116,6 +1116,7 @@ index_create(Relation heapRelation,
else
{
bool have_simple_col = false;
+ bool locked_object = false;
addrs = new_object_addresses();
@@ -1128,6 +1129,12 @@ index_create(Relation heapRelation,
heapRelationId,
indexInfo->ii_IndexAttrNumbers[i]);
add_exact_object_address(&referenced, addrs);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
+ locked_object = true;
+ }
have_simple_col = true;
}
}
@@ -1143,6 +1150,8 @@ index_create(Relation heapRelation,
ObjectAddressSet(referenced, RelationRelationId,
heapRelationId);
add_exact_object_address(&referenced, addrs);
+
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_AUTO);
@@ -1158,9 +1167,13 @@ index_create(Relation heapRelation,
if (OidIsValid(parentIndexRelid))
{
ObjectAddressSet(referenced, RelationRelationId, parentIndexRelid);
+
+ LockNotPinnedObject(RelationRelationId, parentIndexRelid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, heapRelationId);
+
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
@@ -1176,6 +1189,7 @@ index_create(Relation heapRelation,
{
ObjectAddressSet(referenced, CollationRelationId, collationIds[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, collationIds[i]);
}
}
@@ -1184,6 +1198,7 @@ index_create(Relation heapRelation,
{
ObjectAddressSet(referenced, OperatorClassRelationId, opclassIds[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, opclassIds[i]);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -1988,6 +2003,14 @@ index_constraint_create(Relation heapRelation,
*/
ObjectAddressSet(myself, ConstraintRelationId, conOid);
ObjectAddressSet(idxaddr, RelationRelationId, indexRelationId);
+
+ /*
+ * CommandCounterIncrement() here to ensure the new constraint entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+ LockNotPinnedObject(ConstraintRelationId, conOid);
recordDependencyOn(&idxaddr, &myself, DEPENDENCY_INTERNAL);
/*
@@ -1999,9 +2022,12 @@ index_constraint_create(Relation heapRelation,
ObjectAddress referenced;
ObjectAddressSet(referenced, ConstraintRelationId, parentConstraintId);
+ LockNotPinnedObject(ConstraintRelationId, parentConstraintId);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId,
RelationGetRelid(heapRelation));
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(heapRelation));
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 85a7b7e641..17c4085948 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2590,6 +2590,63 @@ get_object_namespace(const ObjectAddress *address)
return oid;
}
+/*
+ * ObjectByIdExist
+ *
+ * Return whether the given object exists.
+ *
+ * Works for most catalogs, if no special processing is needed.
+ */
+bool
+ObjectByIdExist(const ObjectAddress *address)
+{
+ HeapTuple tuple;
+ int cache;
+ const ObjectPropertyType *property;
+
+ property = get_object_property_data(address->classId);
+
+ cache = property->oid_catcache_id;
+
+ if (cache >= 0)
+ {
+ /* Fetch tuple from syscache. */
+ tuple = SearchSysCache1(cache, ObjectIdGetDatum(address->objectId));
+
+ if (!HeapTupleIsValid(tuple))
+ {
+ return false;
+ }
+
+ ReleaseSysCache(tuple);
+
+ return true;
+ }
+ else
+ {
+ Relation rel;
+ ScanKeyData skey[1];
+ SysScanDesc scan;
+
+ rel = table_open(address->classId, AccessShareLock);
+
+ ScanKeyInit(&skey[0],
+ get_object_attnum_oid(address->classId),
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(address->objectId));
+
+ scan = systable_beginscan(rel, get_object_oid_index(address->classId), true,
+ NULL, 1, skey);
+
+ /* we expect exactly one match */
+ tuple = systable_getnext(scan);
+ systable_endscan(scan);
+ table_close(rel, AccessShareLock);
+
+ return (HeapTupleIsValid(tuple));
+ }
+}
+
/*
* Return ObjectType for the given object type as given by
* getObjectTypeDescription; if no valid ObjectType code exists, but it's a
diff --git a/src/backend/catalog/pg_aggregate.c b/src/backend/catalog/pg_aggregate.c
index 90fc7db949..a47e3c5507 100644
--- a/src/backend/catalog/pg_aggregate.c
+++ b/src/backend/catalog/pg_aggregate.c
@@ -748,12 +748,14 @@ AggregateCreate(const char *aggName,
/* Depends on transition function */
ObjectAddressSet(referenced, ProcedureRelationId, transfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, transfn);
/* Depends on final function, if any */
if (OidIsValid(finalfn))
{
ObjectAddressSet(referenced, ProcedureRelationId, finalfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, finalfn);
}
/* Depends on combine function, if any */
@@ -761,6 +763,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, combinefn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, combinefn);
}
/* Depends on serialization function, if any */
@@ -768,6 +771,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, serialfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, serialfn);
}
/* Depends on deserialization function, if any */
@@ -775,6 +779,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, deserialfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, deserialfn);
}
/* Depends on forward transition function, if any */
@@ -782,6 +787,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, mtransfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, mtransfn);
}
/* Depends on inverse transition function, if any */
@@ -789,6 +795,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, minvtransfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, minvtransfn);
}
/* Depends on final function, if any */
@@ -796,6 +803,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, mfinalfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, mfinalfn);
}
/* Depends on sort operator, if any */
@@ -803,6 +811,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, OperatorRelationId, sortop);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorRelationId, sortop);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/catalog/pg_attrdef.c b/src/backend/catalog/pg_attrdef.c
index 003ae70b4d..dcce454f00 100644
--- a/src/backend/catalog/pg_attrdef.c
+++ b/src/backend/catalog/pg_attrdef.c
@@ -178,6 +178,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
colobject.objectId = RelationGetRelid(rel);
colobject.objectSubId = attnum;
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&defobject, &colobject,
attgenerated ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/catalog/pg_cast.c b/src/backend/catalog/pg_cast.c
index 5a5b855d51..d3707e424c 100644
--- a/src/backend/catalog/pg_cast.c
+++ b/src/backend/catalog/pg_cast.c
@@ -97,16 +97,19 @@ CastCreate(Oid sourcetypeid, Oid targettypeid,
/* dependency on source type */
ObjectAddressSet(referenced, TypeRelationId, sourcetypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, sourcetypeid);
/* dependency on target type */
ObjectAddressSet(referenced, TypeRelationId, targettypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, targettypeid);
/* dependency on function */
if (OidIsValid(funcid))
{
ObjectAddressSet(referenced, ProcedureRelationId, funcid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, funcid);
}
/* dependencies on casts required for function */
@@ -114,11 +117,13 @@ CastCreate(Oid sourcetypeid, Oid targettypeid,
{
ObjectAddressSet(referenced, CastRelationId, incastid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CastRelationId, incastid);
}
if (OidIsValid(outcastid))
{
ObjectAddressSet(referenced, CastRelationId, outcastid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CastRelationId, outcastid);
}
record_object_address_dependencies(&myself, addrs, behavior);
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 7f2f701229..78498b8c20 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -218,6 +218,7 @@ CollationCreate(const char *collname, Oid collnamespace,
referenced.classId = NamespaceRelationId;
referenced.objectId = collnamespace;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, collnamespace);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on owner */
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3baf9231ed..c4cdbd7c58 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -252,17 +252,26 @@ CreateConstraintEntry(const char *constraintName,
if (constraintNTotalKeys > 0)
{
+ bool locked_object = false;
+
for (i = 0; i < constraintNTotalKeys; i++)
{
ObjectAddressSubSet(relobject, RelationRelationId, relId,
constraintKey[i]);
add_exact_object_address(&relobject, addrs_auto);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, relId);
+ locked_object = true;
+ }
}
}
else
{
ObjectAddressSet(relobject, RelationRelationId, relId);
add_exact_object_address(&relobject, addrs_auto);
+ LockNotPinnedObject(RelationRelationId, relId);
}
}
@@ -275,6 +284,7 @@ CreateConstraintEntry(const char *constraintName,
ObjectAddressSet(domobject, TypeRelationId, domainId);
add_exact_object_address(&domobject, addrs_auto);
+ LockNotPinnedObject(TypeRelationId, domainId);
}
record_object_address_dependencies(&conobject, addrs_auto,
@@ -294,17 +304,26 @@ CreateConstraintEntry(const char *constraintName,
if (foreignNKeys > 0)
{
+ bool locked_object = false;
+
for (i = 0; i < foreignNKeys; i++)
{
ObjectAddressSubSet(relobject, RelationRelationId,
foreignRelId, foreignKey[i]);
add_exact_object_address(&relobject, addrs_normal);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, foreignRelId);
+ locked_object = true;
+ }
}
}
else
{
ObjectAddressSet(relobject, RelationRelationId, foreignRelId);
add_exact_object_address(&relobject, addrs_normal);
+ LockNotPinnedObject(RelationRelationId, foreignRelId);
}
}
@@ -320,6 +339,7 @@ CreateConstraintEntry(const char *constraintName,
ObjectAddressSet(relobject, RelationRelationId, indexRelId);
add_exact_object_address(&relobject, addrs_normal);
+ LockNotPinnedObject(RelationRelationId, indexRelId);
}
if (foreignNKeys > 0)
@@ -339,15 +359,18 @@ CreateConstraintEntry(const char *constraintName,
{
oprobject.objectId = pfEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, pfEqOp[i]);
if (ppEqOp[i] != pfEqOp[i])
{
oprobject.objectId = ppEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, ppEqOp[i]);
}
if (ffEqOp[i] != pfEqOp[i])
{
oprobject.objectId = ffEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, ffEqOp[i]);
}
}
}
@@ -858,9 +881,12 @@ ConstraintSetParentConstraint(Oid childConstrId,
ObjectAddressSet(depender, ConstraintRelationId, childConstrId);
ObjectAddressSet(referenced, ConstraintRelationId, parentConstrId);
+ LockNotPinnedObject(ConstraintRelationId, parentConstrId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, childTableId);
+
+ LockNotPinnedObject(RelationRelationId, childTableId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_SEC);
}
else
diff --git a/src/backend/catalog/pg_conversion.c b/src/backend/catalog/pg_conversion.c
index 0770878eac..25881654d6 100644
--- a/src/backend/catalog/pg_conversion.c
+++ b/src/backend/catalog/pg_conversion.c
@@ -116,12 +116,14 @@ ConversionCreate(const char *conname, Oid connamespace,
referenced.classId = ProcedureRelationId;
referenced.objectId = conproc;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, conproc);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on namespace */
referenced.classId = NamespaceRelationId;
referenced.objectId = connamespace;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, connamespace);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on owner */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index cfd7ef51df..ebca5a452b 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -20,21 +20,21 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_auth_members.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "catalog/partition.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "storage/lmgr.h"
+#include "storage/lock.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/syscache.h"
#include "utils/rel.h"
-static bool isObjectPinned(const ObjectAddress *object);
-
-
/*
* Record a dependency between 2 objects via their respective objectAddress.
* The first argument is the dependent object, the second the one it
@@ -100,6 +100,37 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+#ifdef USE_ASSERT_CHECKING
+ if (!isObjectPinned(referenced))
+ {
+ if (referenced->classId != RelationRelationId)
+ {
+ LOCKTAG tag;
+
+ SET_LOCKTAG_OBJECT(tag,
+ MyDatabaseId,
+ referenced->classId,
+ referenced->objectId,
+ 0);
+ /* assert the referenced object is locked */
+ Assert(LockHeldByMe(&tag, AccessShareLock, false));
+ }
+ else
+ {
+ Assert(!IsSharedRelation(referenced->objectId));
+
+ /*
+ * Assert the referenced object is locked if it should be
+ * visible (see the comment related to LockNotPinnedObject()
+ * in TypeCreate()).
+ */
+ Assert(!ObjectByIdExist(referenced) ||
+ CheckRelationOidLockedByMe(referenced->objectId,
+ AccessShareLock, true));
+ }
+ }
+#endif
+
/*
* If the referenced object is pinned by the system, there's no real
* need to record dependencies on it. This saves lots of space in
@@ -239,6 +270,7 @@ recordDependencyOnCurrentExtension(const ObjectAddress *object,
extension.objectId = CurrentExtensionObject;
extension.objectSubId = 0;
+ LockNotPinnedObject(ExtensionRelationId, CurrentExtensionObject);
recordDependencyOn(object, &extension, DEPENDENCY_EXTENSION);
}
}
@@ -706,7 +738,7 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
* The passed subId, if any, is ignored; we assume that only whole objects
* are pinned (and that this implies pinning their components).
*/
-static bool
+bool
isObjectPinned(const ObjectAddress *object)
{
return IsPinnedObject(object->classId, object->objectId);
diff --git a/src/backend/catalog/pg_operator.c b/src/backend/catalog/pg_operator.c
index 65b45a424a..e8374eec88 100644
--- a/src/backend/catalog/pg_operator.c
+++ b/src/backend/catalog/pg_operator.c
@@ -251,6 +251,16 @@ OperatorShellMake(const char *operatorName,
values[Anum_pg_operator_oprrest - 1] = ObjectIdGetDatum(InvalidOid);
values[Anum_pg_operator_oprjoin - 1] = ObjectIdGetDatum(InvalidOid);
+ /* Lock dependent objects */
+ if (OidIsValid(operatorNamespace))
+ LockNotPinnedObject(NamespaceRelationId, operatorNamespace);
+
+ if (OidIsValid(leftTypeId))
+ LockNotPinnedObject(TypeRelationId, leftTypeId);
+
+ if (OidIsValid(rightTypeId))
+ LockNotPinnedObject(TypeRelationId, rightTypeId);
+
/*
* create a new operator tuple
*/
@@ -513,6 +523,15 @@ OperatorCreate(const char *operatorName,
CatalogTupleInsert(pg_operator_desc, tup);
}
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, operatorNamespace);
+ LockNotPinnedObject(TypeRelationId, leftTypeId);
+ LockNotPinnedObject(TypeRelationId, rightTypeId);
+ LockNotPinnedObject(TypeRelationId, operResultType);
+ LockNotPinnedObject(ProcedureRelationId, procedureId);
+ LockNotPinnedObject(ProcedureRelationId, restrictionId);
+ LockNotPinnedObject(ProcedureRelationId, joinId);
+
/* Add dependencies for the entry */
address = makeOperatorDependencies(tup, true, isUpdate);
diff --git a/src/backend/catalog/pg_proc.c b/src/backend/catalog/pg_proc.c
index 528c17cd7f..116e524390 100644
--- a/src/backend/catalog/pg_proc.c
+++ b/src/backend/catalog/pg_proc.c
@@ -593,6 +593,13 @@ ProcedureCreate(const char *procedureName,
if (is_update)
deleteDependencyRecordsFor(ProcedureRelationId, retval, true);
+ /*
+ * CommandCounterIncrement() here to ensure the new function entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
addrs = new_object_addresses();
ObjectAddressSet(myself, ProcedureRelationId, retval);
@@ -600,20 +607,24 @@ ProcedureCreate(const char *procedureName,
/* dependency on namespace */
ObjectAddressSet(referenced, NamespaceRelationId, procNamespace);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(NamespaceRelationId, procNamespace);
/* dependency on implementation language */
ObjectAddressSet(referenced, LanguageRelationId, languageObjectId);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(LanguageRelationId, languageObjectId);
/* dependency on return type */
ObjectAddressSet(referenced, TypeRelationId, returnType);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, returnType);
/* dependency on transform used by return type, if any */
if ((trfid = get_transform_oid(returnType, languageObjectId, true)))
{
ObjectAddressSet(referenced, TransformRelationId, trfid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TransformRelationId, trfid);
}
/* dependency on parameter types */
@@ -621,12 +632,14 @@ ProcedureCreate(const char *procedureName,
{
ObjectAddressSet(referenced, TypeRelationId, allParams[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, allParams[i]);
/* dependency on transform used by parameter type, if any */
if ((trfid = get_transform_oid(allParams[i], languageObjectId, true)))
{
ObjectAddressSet(referenced, TransformRelationId, trfid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TransformRelationId, trfid);
}
}
@@ -635,6 +648,7 @@ ProcedureCreate(const char *procedureName,
{
ObjectAddressSet(referenced, ProcedureRelationId, prosupport);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, prosupport);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -674,9 +688,6 @@ ProcedureCreate(const char *procedureName,
ArrayType *set_items = NULL;
int save_nestlevel = 0;
- /* Advance command counter so new tuple can be seen by validator */
- CommandCounterIncrement();
-
/*
* Set per-function configuration parameters so that the validation is
* done with the environment the function expects. However, if
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2b86..d533e4fd31 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -395,6 +395,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
referenced;
List *relids = NIL;
int i;
+ bool locked = false;
rel = table_open(PublicationRelRelationId, RowExclusiveLock);
@@ -457,10 +458,13 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
/* Add dependency on the publication */
ObjectAddressSet(referenced, PublicationRelationId, pubid);
+ LockNotPinnedObject(PublicationRelationId, pubid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the relation */
ObjectAddressSet(referenced, RelationRelationId, relid);
+
+ LockNotPinnedObject(RelationRelationId, relid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the objects mentioned in the qualifications */
@@ -474,6 +478,11 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
while ((i = bms_next_member(attnums, i)) >= 0)
{
ObjectAddressSubSet(referenced, RelationRelationId, relid, i);
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, relid);
+ locked = true;
+ }
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -643,10 +652,12 @@ publication_add_schema(Oid pubid, Oid schemaid, bool if_not_exists)
/* Add dependency on the publication */
ObjectAddressSet(referenced, PublicationRelationId, pubid);
+ LockNotPinnedObject(PublicationRelationId, pubid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the schema */
ObjectAddressSet(referenced, NamespaceRelationId, schemaid);
+ LockNotPinnedObject(NamespaceRelationId, schemaid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Close the table */
diff --git a/src/backend/catalog/pg_range.c b/src/backend/catalog/pg_range.c
index 501a6ba410..e5b5a0b6f8 100644
--- a/src/backend/catalog/pg_range.c
+++ b/src/backend/catalog/pg_range.c
@@ -70,26 +70,31 @@ RangeCreate(Oid rangeTypeOid, Oid rangeSubType, Oid rangeCollation,
ObjectAddressSet(referenced, TypeRelationId, rangeSubType);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, rangeSubType);
ObjectAddressSet(referenced, OperatorClassRelationId, rangeSubOpclass);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, rangeSubOpclass);
if (OidIsValid(rangeCollation))
{
ObjectAddressSet(referenced, CollationRelationId, rangeCollation);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, rangeCollation);
}
if (OidIsValid(rangeCanonical))
{
ObjectAddressSet(referenced, ProcedureRelationId, rangeCanonical);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, rangeCanonical);
}
if (OidIsValid(rangeSubDiff))
{
ObjectAddressSet(referenced, ProcedureRelationId, rangeSubDiff);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, rangeSubDiff);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -99,6 +104,7 @@ RangeCreate(Oid rangeTypeOid, Oid rangeSubType, Oid rangeCollation,
referencing.classId = TypeRelationId;
referencing.objectId = multirangeTypeOid;
referencing.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, rangeTypeOid);
recordDependencyOn(&referencing, &myself, DEPENDENCY_INTERNAL);
table_close(pg_range, RowExclusiveLock);
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 395dec8ed8..82ee7bc2e3 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -157,6 +157,12 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
* Create dependencies. We can/must skip this in bootstrap mode.
*/
if (!IsBootstrapProcessingMode())
+ {
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, typeNamespace);
+ LockNotPinnedObject(ProcedureRelationId, F_SHELL_IN);
+ LockNotPinnedObject(ProcedureRelationId, F_SHELL_OUT);
+
GenerateTypeDependencies(tup,
pg_type_desc,
NULL,
@@ -166,6 +172,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
false,
true, /* make extension dependency */
false);
+ }
/* Post creation hook for new shell type */
InvokeObjectPostCreateHook(TypeRelationId, typoid, 0);
@@ -494,6 +501,37 @@ TypeCreate(Oid newTypeOid,
* Create dependencies. We can/must skip this in bootstrap mode.
*/
if (!IsBootstrapProcessingMode())
+ {
+ /*
+ * CommandCounterIncrement() here to ensure the new type entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, typeNamespace);
+ LockNotPinnedObject(ProcedureRelationId, inputProcedure);
+ LockNotPinnedObject(ProcedureRelationId, outputProcedure);
+ LockNotPinnedObject(ProcedureRelationId, receiveProcedure);
+ LockNotPinnedObject(ProcedureRelationId, sendProcedure);
+ LockNotPinnedObject(ProcedureRelationId, typmodinProcedure);
+ LockNotPinnedObject(ProcedureRelationId, typmodoutProcedure);
+ LockNotPinnedObject(ProcedureRelationId, analyzeProcedure);
+ LockNotPinnedObject(ProcedureRelationId, subscriptProcedure);
+ LockNotPinnedObject(TypeRelationId, baseType);
+ LockNotPinnedObject(CollationRelationId, typeCollation);
+ LockNotPinnedObject(TypeRelationId, elementType);
+
+ /*
+ * No need to call LockRelationOid() (through LockNotPinnedObject())
+ * on relationOid as relationOid is set to an InvalidOid or to a new
+ * Oid not added to pg_class yet (In heap_create_with_catalog(),
+ * AddNewRelationType() is called before AddNewRelationTuple()).
+ */
+ if (relationKind == RELKIND_COMPOSITE_TYPE)
+ LockNotPinnedObject(TypeRelationId, typeObjectId);
+
GenerateTypeDependencies(tup,
pg_type_desc,
(defaultTypeBin ?
@@ -505,6 +543,7 @@ TypeCreate(Oid newTypeOid,
isDependentType,
true, /* make extension dependency */
rebuildDeps);
+ }
/* Post creation hook for new type */
InvokeObjectPostCreateHook(TypeRelationId, typeObjectId, 0);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 738bc46ae8..a4d8342ca1 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -367,6 +367,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
toastobject.objectId = toast_relid;
toastobject.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, relOid);
recordDependencyOn(&toastobject, &baseobject, DEPENDENCY_INTERNAL);
}
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f99ebb447..57e86f576a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -501,7 +501,10 @@ ExecAlterObjectDependsStmt(AlterObjectDependsStmt *stmt, ObjectAddress *refAddre
currexts = getAutoExtensionsOfObject(address.classId,
address.objectId);
if (!list_member_oid(currexts, refAddr.objectId))
+ {
+ LockNotPinnedObject(refAddr.classId, refAddr.objectId);
recordDependencyOn(&address, &refAddr, DEPENDENCY_AUTO_EXTENSION);
+ }
}
return address;
@@ -807,6 +810,7 @@ AlterObjectNamespace_internal(Relation rel, Oid objid, Oid nspOid)
pfree(replaces);
/* update dependency to point to the new schema */
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(classId, objid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for object %u",
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index aaa0f9a1dc..8616a7c9fa 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -104,6 +104,7 @@ CreateAccessMethod(CreateAmStmt *stmt)
referenced.objectId = amhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, amhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
recordDependencyOnCurrentExtension(&myself, false);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 78f96789b0..fb95d17738 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1272,6 +1272,7 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
*/
if (relam1 != relam2)
{
+ LockNotPinnedObject(AccessMethodRelationId, relam2);
if (changeDependencyFor(RelationRelationId,
r1,
AccessMethodRelationId,
@@ -1280,6 +1281,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
elog(ERROR, "could not change access method dependency for relation \"%s.%s\"",
get_namespace_name(get_rel_namespace(r1)),
get_rel_name(r1));
+
+ LockNotPinnedObject(AccessMethodRelationId, relam1);
if (changeDependencyFor(RelationRelationId,
r2,
AccessMethodRelationId,
@@ -1381,6 +1384,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
{
baseobject.objectId = r1;
toastobject.objectId = relform1->reltoastrelid;
+
+ LockNotPinnedObject(RelationRelationId, r1);
recordDependencyOn(&toastobject, &baseobject,
DEPENDENCY_INTERNAL);
}
@@ -1389,6 +1394,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
{
baseobject.objectId = r2;
toastobject.objectId = relform2->reltoastrelid;
+
+ LockNotPinnedObject(RelationRelationId, r2);
recordDependencyOn(&toastobject, &baseobject,
DEPENDENCY_INTERNAL);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 7a5ed6b985..8d0cdec59e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -327,6 +327,7 @@ insert_event_trigger_tuple(const char *trigname, const char *eventname, Oid evtO
referenced.classId = ProcedureRelationId;
referenced.objectId = funcoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, funcoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* Depend on extension, if any. */
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 1643c8c69a..669a5d6dd8 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -1924,6 +1924,7 @@ InsertExtensionTuple(const char *extName, Oid extOwner,
ObjectAddressSet(nsp, NamespaceRelationId, schemaOid);
add_exact_object_address(&nsp, refobjs);
+ LockNotPinnedObject(NamespaceRelationId, schemaOid);
foreach(lc, requiredExtensions)
{
@@ -1932,6 +1933,7 @@ InsertExtensionTuple(const char *extName, Oid extOwner,
ObjectAddressSet(otherext, ExtensionRelationId, reqext);
add_exact_object_address(&otherext, refobjs);
+ LockNotPinnedObject(ExtensionRelationId, reqext);
}
/* Record all of them (this includes duplicate elimination) */
@@ -2968,6 +2970,7 @@ AlterExtensionNamespace(const char *extensionName, const char *newschema, Oid *o
table_close(extRel, RowExclusiveLock);
/* update dependency to point to the new schema */
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(ExtensionRelationId, extensionOid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for extension %s",
@@ -3258,6 +3261,7 @@ ApplyExtensionUpdates(Oid extensionOid,
otherext.objectId = reqext;
otherext.objectSubId = 0;
+ LockNotPinnedObject(ExtensionRelationId, reqext);
recordDependencyOn(&myself, &otherext, DEPENDENCY_NORMAL);
}
@@ -3414,6 +3418,7 @@ ExecAlterExtensionContentsRecurse(AlterExtensionContentsStmt *stmt,
/*
* OK, add the dependency.
*/
+ LockNotPinnedObject(extension.classId, extension.objectId);
recordDependencyOn(&object, &extension, DEPENDENCY_EXTENSION);
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index cf61bbac1f..735bca486c 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -642,6 +642,7 @@ CreateForeignDataWrapper(ParseState *pstate, CreateFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -650,6 +651,7 @@ CreateForeignDataWrapper(ParseState *pstate, CreateFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwvalidator;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwvalidator);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -811,6 +813,7 @@ AlterForeignDataWrapper(ParseState *pstate, AlterFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -819,6 +822,7 @@ AlterForeignDataWrapper(ParseState *pstate, AlterFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwvalidator;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwvalidator);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -951,6 +955,7 @@ CreateForeignServer(CreateForeignServerStmt *stmt)
referenced.classId = ForeignDataWrapperRelationId;
referenced.objectId = fdw->fdwid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignDataWrapperRelationId, fdw->fdwid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
recordDependencyOnOwner(ForeignServerRelationId, srvId, ownerId);
@@ -1195,6 +1200,7 @@ CreateUserMapping(CreateUserMappingStmt *stmt)
referenced.classId = ForeignServerRelationId;
referenced.objectId = srv->serverid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignServerRelationId, srv->serverid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
if (OidIsValid(useId))
@@ -1472,6 +1478,7 @@ CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid)
referenced.classId = ForeignServerRelationId;
referenced.objectId = server->serverid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignServerRelationId, server->serverid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
table_close(ftrel, RowExclusiveLock);
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 6593fd7d81..8207ef08b3 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -1446,6 +1446,7 @@ AlterFunction(ParseState *pstate, AlterFunctionStmt *stmt)
/* Add or replace dependency on support function */
if (OidIsValid(procForm->prosupport))
{
+ LockNotPinnedObject(ProcedureRelationId, newsupport);
if (changeDependencyFor(ProcedureRelationId, funcOid,
ProcedureRelationId, procForm->prosupport,
newsupport) != 1)
@@ -1459,6 +1460,7 @@ AlterFunction(ParseState *pstate, AlterFunctionStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = newsupport;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, newsupport);
recordDependencyOn(&address, &referenced, DEPENDENCY_NORMAL);
}
@@ -1962,21 +1964,25 @@ CreateTransform(CreateTransformStmt *stmt)
/* dependency on language */
ObjectAddressSet(referenced, LanguageRelationId, langid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(LanguageRelationId, langid);
/* dependency on type */
ObjectAddressSet(referenced, TypeRelationId, typeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, typeid);
/* dependencies on functions */
if (OidIsValid(fromsqlfuncid))
{
ObjectAddressSet(referenced, ProcedureRelationId, fromsqlfuncid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, fromsqlfuncid);
}
if (OidIsValid(tosqlfuncid))
{
ObjectAddressSet(referenced, ProcedureRelationId, tosqlfuncid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, tosqlfuncid);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index c5a56c75f6..fc7a28baf9 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -4384,8 +4384,10 @@ IndexSetParentIndex(Relation partitionIdx, Oid parentOid)
ObjectAddressSet(parentIdx, RelationRelationId, parentOid);
ObjectAddressSet(partitionTbl, RelationRelationId,
partitionIdx->rd_index->indrelid);
+ LockNotPinnedObject(RelationRelationId, parentOid);
recordDependencyOn(&partIdx, &parentIdx,
DEPENDENCY_PARTITION_PRI);
+ LockNotPinnedObject(RelationRelationId, partitionIdx->rd_index->indrelid);
recordDependencyOn(&partIdx, &partitionTbl,
DEPENDENCY_PARTITION_SEC);
}
diff --git a/src/backend/commands/opclasscmds.c b/src/backend/commands/opclasscmds.c
index b8b5c147c5..e70afd216c 100644
--- a/src/backend/commands/opclasscmds.c
+++ b/src/backend/commands/opclasscmds.c
@@ -298,12 +298,14 @@ CreateOpFamily(CreateOpFamilyStmt *stmt, const char *opfname,
referenced.classId = AccessMethodRelationId;
referenced.objectId = amoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(AccessMethodRelationId, amoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* dependency on namespace */
referenced.classId = NamespaceRelationId;
referenced.objectId = namespaceoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on owner */
@@ -725,18 +727,21 @@ DefineOpClass(CreateOpClassStmt *stmt)
referenced.classId = NamespaceRelationId;
referenced.objectId = namespaceoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on opfamily */
referenced.classId = OperatorFamilyRelationId;
referenced.objectId = opfamilyoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(OperatorFamilyRelationId, opfamilyoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* dependency on indexed datatype */
referenced.classId = TypeRelationId;
referenced.objectId = typeoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, typeoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on storage datatype */
@@ -745,6 +750,7 @@ DefineOpClass(CreateOpClassStmt *stmt)
referenced.classId = TypeRelationId;
referenced.objectId = storageoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, storageoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -1486,6 +1492,13 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
heap_freetuple(tup);
+ /*
+ * CommandCounterIncrement() here to ensure the new operator entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
/* Make its dependencies */
myself.classId = AccessMethodOperatorRelationId;
myself.objectId = entryoid;
@@ -1496,6 +1509,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectSubId = 0;
/* see comments in amapi.h about dependency strength */
+ LockNotPinnedObject(OperatorRelationId, op->object);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
@@ -1504,6 +1518,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = op->refobjid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(referenced.classId, op->refobjid);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
@@ -1514,6 +1529,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = op->sortfamily;
referenced.objectSubId = 0;
+ LockNotPinnedObject(OperatorFamilyRelationId, op->sortfamily);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
}
@@ -1597,6 +1613,7 @@ storeProcedures(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectSubId = 0;
/* see comments in amapi.h about dependency strength */
+ LockNotPinnedObject(ProcedureRelationId, proc->object);
recordDependencyOn(&myself, &referenced,
proc->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
@@ -1605,6 +1622,7 @@ storeProcedures(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = proc->refobjid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(referenced.classId, proc->refobjid);
recordDependencyOn(&myself, &referenced,
proc->ref_is_hard ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/commands/operatorcmds.c b/src/backend/commands/operatorcmds.c
index 5872a3e192..58a69e7cc2 100644
--- a/src/backend/commands/operatorcmds.c
+++ b/src/backend/commands/operatorcmds.c
@@ -33,6 +33,7 @@
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_namespace.h"
@@ -656,11 +657,15 @@ AlterOperator(AlterOperatorStmt *stmt)
{
replaces[Anum_pg_operator_oprrest - 1] = true;
values[Anum_pg_operator_oprrest - 1] = ObjectIdGetDatum(restrictionOid);
+ if (OidIsValid(restrictionOid))
+ LockNotPinnedObject(ProcedureRelationId, restrictionOid);
}
if (updateJoin)
{
replaces[Anum_pg_operator_oprjoin - 1] = true;
values[Anum_pg_operator_oprjoin - 1] = ObjectIdGetDatum(joinOid);
+ if (OidIsValid(joinOid))
+ LockNotPinnedObject(ProcedureRelationId, joinOid);
}
if (OidIsValid(commutatorOid))
{
@@ -688,6 +693,31 @@ AlterOperator(AlterOperatorStmt *stmt)
CatalogTupleUpdate(catalog, &tup->t_self, tup);
+
+ /* Lock dependent objects */
+ oprForm = (Form_pg_operator) GETSTRUCT(tup);
+
+ if (OidIsValid(oprForm->oprnamespace))
+ LockNotPinnedObject(NamespaceRelationId, oprForm->oprnamespace);
+
+ if (OidIsValid(oprForm->oprleft))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprleft);
+
+ if (OidIsValid(oprForm->oprright))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprright);
+
+ if (OidIsValid(oprForm->oprresult))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprresult);
+
+ if (OidIsValid(oprForm->oprcode))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprcode);
+
+ if (OidIsValid(oprForm->oprrest))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprrest);
+
+ if (OidIsValid(oprForm->oprjoin))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprjoin);
+
address = makeOperatorDependencies(tup, false, true);
if (OidIsValid(commutatorOid) || OidIsValid(negatorOid))
diff --git a/src/backend/commands/policy.c b/src/backend/commands/policy.c
index 6ff3eba824..9da98cbeec 100644
--- a/src/backend/commands/policy.c
+++ b/src/backend/commands/policy.c
@@ -722,6 +722,7 @@ CreatePolicy(CreatePolicyStmt *stmt)
myself.objectId = policy_id;
myself.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, table_id);
recordDependencyOn(&myself, &target, DEPENDENCY_AUTO);
recordDependencyOnExpr(&myself, qual, qual_pstate->p_rtable,
@@ -1053,6 +1054,7 @@ AlterPolicy(AlterPolicyStmt *stmt)
myself.objectId = policy_id;
myself.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, table_id);
recordDependencyOn(&myself, &target, DEPENDENCY_AUTO);
recordDependencyOnExpr(&myself, qual, qual_parse_rtable, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/proclang.c b/src/backend/commands/proclang.c
index 881f90017e..fadfd9064f 100644
--- a/src/backend/commands/proclang.c
+++ b/src/backend/commands/proclang.c
@@ -190,12 +190,14 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
/* dependency on the PL handler function */
ObjectAddressSet(referenced, ProcedureRelationId, handlerOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, handlerOid);
/* dependency on the inline handler function, if any */
if (OidIsValid(inlineOid))
{
ObjectAddressSet(referenced, ProcedureRelationId, inlineOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, inlineOid);
}
/* dependency on the validator function, if any */
@@ -203,6 +205,7 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
{
ObjectAddressSet(referenced, ProcedureRelationId, valOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, valOid);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 8c1131f020..b94677ae65 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1688,6 +1688,8 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
depobject.classId = RelationRelationId;
depobject.objectId = RelationGetRelid(seqrel);
depobject.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(tablerel));
recordDependencyOn(&depobject, &refobject, deptype);
}
diff --git a/src/backend/commands/statscmds.c b/src/backend/commands/statscmds.c
index 1db3ef69d2..9f0b03388a 100644
--- a/src/backend/commands/statscmds.c
+++ b/src/backend/commands/statscmds.c
@@ -88,6 +88,7 @@ CreateStatistics(CreateStatsStmt *stmt)
bool build_mcv;
bool build_expressions;
bool requested_type = false;
+ bool locked_object = false;
int i;
ListCell *cell;
ListCell *cell2;
@@ -536,6 +537,12 @@ CreateStatistics(CreateStatsStmt *stmt)
for (i = 0; i < nattnums; i++)
{
ObjectAddressSubSet(parentobject, RelationRelationId, relid, attnums[i]);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, relid);
+ locked_object = true;
+ }
recordDependencyOn(&myself, &parentobject, DEPENDENCY_AUTO);
}
@@ -553,6 +560,8 @@ CreateStatistics(CreateStatsStmt *stmt)
if (!nattnums)
{
ObjectAddressSet(parentobject, RelationRelationId, relid);
+
+ LockNotPinnedObject(RelationRelationId, relid);
recordDependencyOn(&myself, &parentobject, DEPENDENCY_AUTO);
}
@@ -573,6 +582,7 @@ CreateStatistics(CreateStatsStmt *stmt)
* than the underlying table(s).
*/
ObjectAddressSet(parentobject, NamespaceRelationId, namespaceId);
+ LockNotPinnedObject(NamespaceRelationId, namespaceId);
recordDependencyOn(&myself, &parentobject, DEPENDENCY_NORMAL);
recordDependencyOnOwner(StatisticExtRelationId, statoid, stxowner);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7a36db6af6..ff605bd0ff 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3419,6 +3419,7 @@ StoreCatalogInheritance1(Oid relationId, Oid parentOid,
childobject.objectId = relationId;
childobject.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, parentOid);
recordDependencyOn(&childobject, &parentobject,
child_dependency_type(child_is_partition));
@@ -7349,7 +7350,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/*
* Add needed dependency entries for the new column.
*/
+ LockNotPinnedObject(TypeRelationId, attribute->atttypid);
add_column_datatype_dependency(myrelid, newattnum, attribute->atttypid);
+ LockNotPinnedObject(CollationRelationId, attribute->attcollation);
add_column_collation_dependency(myrelid, newattnum, attribute->attcollation);
/*
@@ -10174,6 +10177,7 @@ addFkRecurseReferenced(List **wqueue, Constraint *fkconstraint, Relation rel,
ObjectAddress referenced;
ObjectAddressSet(referenced, ConstraintRelationId, parentConstr);
+ LockNotPinnedObject(ConstraintRelationId, parentConstr);
recordDependencyOn(&address, &referenced, DEPENDENCY_INTERNAL);
}
@@ -10465,8 +10469,11 @@ addFkRecurseReferencing(List **wqueue, Constraint *fkconstraint, Relation rel,
*/
ObjectAddressSet(address, ConstraintRelationId, constrOid);
ObjectAddressSet(referenced, ConstraintRelationId, parentConstr);
+ LockNotPinnedObject(ConstraintRelationId, parentConstr);
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, partitionId);
+
+ LockNotPinnedObject(RelationRelationId, partitionId);
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_SEC);
/* Make all this visible before recursing */
@@ -10984,9 +10991,12 @@ CloneFkReferencing(List **wqueue, Relation parentRel, Relation partRel)
/* Set up partition dependencies for the new constraint */
ObjectAddressSet(address, ConstraintRelationId, constrOid);
ObjectAddressSet(referenced, ConstraintRelationId, parentConstrOid);
+ LockDatabaseObject(ConstraintRelationId, parentConstrOid, 0, AccessShareLock);
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId,
RelationGetRelid(partRel));
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(partRel));
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_SEC);
/* Done with the cloned constraint's tuple */
@@ -13272,7 +13282,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
table_close(attrelation, RowExclusiveLock);
/* Install dependencies on new datatype and collation */
+ LockNotPinnedObject(TypeRelationId, targettype);
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
+ LockNotPinnedObject(CollationRelationId, targetcollid);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
/*
@@ -14834,6 +14846,7 @@ ATExecSetAccessMethodNoStorage(Relation rel, Oid newAccessMethodId)
*/
ObjectAddressSet(relobj, RelationRelationId, reloid);
ObjectAddressSet(referenced, AccessMethodRelationId, rd_rel->relam);
+ LockNotPinnedObject(AccessMethodRelationId, rd_rel->relam);
recordDependencyOn(&relobj, &referenced, DEPENDENCY_NORMAL);
}
else if (OidIsValid(oldAccessMethodId) &&
@@ -14853,6 +14866,7 @@ ATExecSetAccessMethodNoStorage(Relation rel, Oid newAccessMethodId)
OidIsValid(rd_rel->relam));
/* Both are valid, so update the dependency */
+ LockNotPinnedObject(AccessMethodRelationId, rd_rel->relam);
changeDependencyFor(RelationRelationId, reloid,
AccessMethodRelationId,
oldAccessMethodId, rd_rel->relam);
@@ -16452,6 +16466,7 @@ ATExecAddOf(Relation rel, const TypeName *ofTypename, LOCKMODE lockmode)
typeobj.classId = TypeRelationId;
typeobj.objectId = typeid;
typeobj.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, typeid);
recordDependencyOn(&tableobj, &typeobj, DEPENDENCY_NORMAL);
/* Update pg_class.reloftype */
@@ -17210,14 +17225,17 @@ AlterRelationNamespaceInternal(Relation classRel, Oid relOid,
CatalogTupleUpdate(classRel, &classTup->t_self, classTup);
/* Update dependency on schema if caller said so */
- if (hasDependEntry &&
- changeDependencyFor(RelationRelationId,
- relOid,
- NamespaceRelationId,
- oldNspOid,
- newNspOid) != 1)
- elog(ERROR, "could not change schema dependency for relation \"%s\"",
- NameStr(classForm->relname));
+ if (hasDependEntry)
+ {
+ LockNotPinnedObject(NamespaceRelationId, newNspOid);
+ if (changeDependencyFor(RelationRelationId,
+ relOid,
+ NamespaceRelationId,
+ oldNspOid,
+ newNspOid) != 1)
+ elog(ERROR, "could not change schema dependency for relation \"%s\"",
+ NameStr(classForm->relname));
+ }
}
if (!already_done)
{
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 170360edda..1250673b16 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -1018,8 +1018,6 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
((Form_pg_class) GETSTRUCT(tuple))->relhastriggers = true;
CatalogTupleUpdate(pgrel, &tuple->t_self, tuple);
-
- CommandCounterIncrement();
}
else
CacheInvalidateRelcacheByTuple(tuple);
@@ -1027,6 +1025,13 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
heap_freetuple(tuple);
table_close(pgrel, RowExclusiveLock);
+ /*
+ * CommandCounterIncrement() here to ensure the new trigger entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
/*
* If we're replacing a trigger, flush all the old dependencies before
* recording new ones.
@@ -1045,6 +1050,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ProcedureRelationId;
referenced.objectId = funcoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, funcoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
if (isInternal && OidIsValid(constraintOid))
@@ -1058,6 +1064,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ConstraintRelationId;
referenced.objectId = constraintOid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ConstraintRelationId, constraintOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
}
else
@@ -1070,6 +1077,8 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = RelationRelationId;
referenced.objectId = RelationGetRelid(rel);
referenced.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
if (OidIsValid(constrrelid))
@@ -1077,6 +1086,8 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = RelationRelationId;
referenced.objectId = constrrelid;
referenced.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, constrrelid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
}
/* Not possible to have an index dependency in this case */
@@ -1091,6 +1102,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ConstraintRelationId;
referenced.objectId = constraintOid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TriggerRelationId, trigoid);
recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
}
@@ -1100,8 +1112,11 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
if (OidIsValid(parentTriggerOid))
{
ObjectAddressSet(referenced, TriggerRelationId, parentTriggerOid);
+ LockNotPinnedObject(TriggerRelationId, parentTriggerOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, RelationGetRelid(rel));
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
}
@@ -1110,12 +1125,19 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
if (columns != NULL)
{
int i;
+ bool locked_object = false;
referenced.classId = RelationRelationId;
referenced.objectId = RelationGetRelid(rel);
for (i = 0; i < ncolumns; i++)
{
referenced.objectSubId = columns[i];
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
+ locked_object = true;
+ }
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -1255,9 +1277,12 @@ TriggerSetParentTrigger(Relation trigRel,
ObjectAddressSet(depender, TriggerRelationId, childTrigId);
ObjectAddressSet(referenced, TriggerRelationId, parentTrigId);
+ LockNotPinnedObject(TriggerRelationId, parentTrigId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, childTableId);
+
+ LockNotPinnedObject(RelationRelationId, childTableId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_SEC);
}
else
diff --git a/src/backend/commands/tsearchcmds.c b/src/backend/commands/tsearchcmds.c
index b7b5019f1e..1b90e187ea 100644
--- a/src/backend/commands/tsearchcmds.c
+++ b/src/backend/commands/tsearchcmds.c
@@ -214,6 +214,7 @@ DefineTSParser(List *names, List *parameters)
namestrcpy(&pname, prsname);
values[Anum_pg_ts_parser_prsname - 1] = NameGetDatum(&pname);
values[Anum_pg_ts_parser_prsnamespace - 1] = ObjectIdGetDatum(namespaceoid);
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
/*
* loop over the definition list and extract the information we need.
@@ -224,28 +225,48 @@ DefineTSParser(List *names, List *parameters)
if (strcmp(defel->defname, "start") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsstart - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsstart);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsstart - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "gettoken") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prstoken - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prstoken);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prstoken - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "end") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsend - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsend);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsend - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "headline") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsheadline - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsheadline);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsheadline - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "lextypes") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prslextype - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prslextype);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prslextype - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else
ereport(ERROR,
@@ -474,6 +495,10 @@ DefineTSDictionary(List *names, List *parameters)
CatalogTupleInsert(dictRel, tup);
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
+ LockNotPinnedObject(TSTemplateRelationId, templId);
+
address = makeDictionaryDependencies(tup);
/* Post creation hook for new text search dictionary */
@@ -723,6 +748,7 @@ DefineTSTemplate(List *names, List *parameters)
namestrcpy(&dname, tmplname);
values[Anum_pg_ts_template_tmplname - 1] = NameGetDatum(&dname);
values[Anum_pg_ts_template_tmplnamespace - 1] = ObjectIdGetDatum(namespaceoid);
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
/*
* loop over the definition list and extract the information we need.
@@ -733,15 +759,23 @@ DefineTSTemplate(List *names, List *parameters)
if (strcmp(defel->defname, "init") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_template_tmplinit - 1] =
get_ts_template_func(defel, Anum_pg_ts_template_tmplinit);
nulls[Anum_pg_ts_template_tmplinit - 1] = false;
+ procoid = DatumGetObjectId(values[Anum_pg_ts_template_tmplinit - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "lexize") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_template_tmpllexize - 1] =
get_ts_template_func(defel, Anum_pg_ts_template_tmpllexize);
nulls[Anum_pg_ts_template_tmpllexize - 1] = false;
+ procoid = DatumGetObjectId(values[Anum_pg_ts_template_tmpllexize - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else
ereport(ERROR,
@@ -998,6 +1032,10 @@ DefineTSConfiguration(List *names, List *parameters, ObjectAddress *copied)
values[Anum_pg_ts_config_cfgowner - 1] = ObjectIdGetDatum(GetUserId());
values[Anum_pg_ts_config_cfgparser - 1] = ObjectIdGetDatum(prsOid);
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
+ LockNotPinnedObject(TSParserRelationId, prsOid);
+
tup = heap_form_tuple(cfgRel->rd_att, values, nulls);
CatalogTupleInsert(cfgRel, tup);
@@ -1063,6 +1101,7 @@ DefineTSConfiguration(List *names, List *parameters, ObjectAddress *copied)
slot[slot_stored_count]->tts_values[Anum_pg_ts_config_map_mapseqno - 1] = cfgmap->mapseqno;
slot[slot_stored_count]->tts_values[Anum_pg_ts_config_map_mapdict - 1] = cfgmap->mapdict;
+ LockNotPinnedObject(TSDictionaryRelationId, cfgmap->mapdict);
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
@@ -1156,9 +1195,13 @@ ObjectAddress
AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
{
HeapTuple tup;
+ Form_pg_ts_config cfg;
Oid cfgId;
Relation relMap;
ObjectAddress address;
+ ScanKeyData skey;
+ SysScanDesc scan;
+ HeapTuple maptup;
/* Find the configuration */
tup = GetTSConfigTuple(stmt->cfgname);
@@ -1168,7 +1211,8 @@ AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
errmsg("text search configuration \"%s\" does not exist",
NameListToString(stmt->cfgname))));
- cfgId = ((Form_pg_ts_config) GETSTRUCT(tup))->oid;
+ cfg = (Form_pg_ts_config) GETSTRUCT(tup);
+ cfgId = cfg->oid;
/* must be owner */
if (!object_ownercheck(TSConfigRelationId, cfgId, GetUserId()))
@@ -1183,6 +1227,28 @@ AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
else if (stmt->tokentype)
DropConfigurationMapping(stmt, tup, relMap);
+ /* Lock dependent objects */
+
+ ScanKeyInit(&skey,
+ Anum_pg_ts_config_map_mapcfg,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(cfgId));
+
+ scan = systable_beginscan(relMap, TSConfigMapIndexId, true,
+ NULL, 1, &skey);
+
+ while (HeapTupleIsValid((maptup = systable_getnext(scan))))
+ {
+ Form_pg_ts_config_map cfgmap = (Form_pg_ts_config_map) GETSTRUCT(maptup);
+
+ LockNotPinnedObject(TSDictionaryRelationId, cfgmap->mapdict);
+ }
+
+ systable_endscan(scan);
+
+ LockNotPinnedObject(NamespaceRelationId, cfg->cfgnamespace);
+ LockNotPinnedObject(TSParserRelationId, cfg->cfgparser);
+
/* Update dependencies */
makeConfigurationDependencies(tup, true, relMap);
@@ -1414,6 +1480,8 @@ MakeConfigurationMapping(AlterTSConfigurationStmt *stmt,
repl_val[Anum_pg_ts_config_map_mapdict - 1] = ObjectIdGetDatum(dictNew);
repl_repl[Anum_pg_ts_config_map_mapdict - 1] = true;
+ LockNotPinnedObject(TSDictionaryRelationId, dictNew);
+
newtup = heap_modify_tuple(maptup,
RelationGetDescr(relMap),
repl_val, repl_null, repl_repl);
@@ -1456,6 +1524,9 @@ MakeConfigurationMapping(AlterTSConfigurationStmt *stmt,
slot[slotCount]->tts_values[Anum_pg_ts_config_map_mapseqno - 1] = Int32GetDatum(j + 1);
slot[slotCount]->tts_values[Anum_pg_ts_config_map_mapdict - 1] = ObjectIdGetDatum(dictIds[j]);
+ /* Lock dependent objects */
+ LockNotPinnedObject(TSDictionaryRelationId, dictIds[j]);
+
ExecStoreVirtualTuple(slot[slotCount]);
slotCount++;
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 2a1e713335..9febaa24a7 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -1794,6 +1794,7 @@ makeRangeConstructors(const char *name, Oid namespace,
* that they go away silently when the type is dropped. Note that
* pg_dump depends on this choice to avoid dumping the constructors.
*/
+ LockNotPinnedObject(TypeRelationId, rangeOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
}
}
@@ -1859,6 +1860,7 @@ makeMultirangeConstructors(const char *name, Oid namespace,
* that they go away silently when the type is dropped. Note that pg_dump
* depends on this choice to avoid dumping the constructors.
*/
+ LockNotPinnedObject(TypeRelationId, multirangeOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
pfree(argtypes);
@@ -2672,6 +2674,45 @@ AlterDomainDefault(List *names, Node *defaultRaw)
CatalogTupleUpdate(rel, &tup->t_self, newtuple);
+ /* Lock dependent objects */
+ typTup = (Form_pg_type) GETSTRUCT(newtuple);
+
+ if (OidIsValid(typTup->typnamespace))
+ LockNotPinnedObject(NamespaceRelationId, typTup->typnamespace);
+
+ if (OidIsValid(typTup->typinput))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typinput);
+
+ if (OidIsValid(typTup->typoutput))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typoutput);
+
+ if (OidIsValid(typTup->typreceive))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typreceive);
+
+ if (OidIsValid(typTup->typsend))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typsend);
+
+ if (OidIsValid(typTup->typmodin))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typmodin);
+
+ if (OidIsValid(typTup->typmodout))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typmodout);
+
+ if (OidIsValid(typTup->typanalyze))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typanalyze);
+
+ if (OidIsValid(typTup->typsubscript))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typsubscript);
+
+ if (OidIsValid(typTup->typbasetype))
+ LockNotPinnedObject(TypeRelationId, typTup->typbasetype);
+
+ if (OidIsValid(typTup->typcollation))
+ LockNotPinnedObject(CollationRelationId, typTup->typcollation);
+
+ if (OidIsValid(typTup->typelem))
+ LockNotPinnedObject(TypeRelationId, typTup->typelem);
+
/* Rebuild dependencies */
GenerateTypeDependencies(newtuple,
rel,
@@ -4276,10 +4317,13 @@ AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
if (oldNspOid != nspOid &&
(isCompositeType || typform->typtype != TYPTYPE_COMPOSITE) &&
!isImplicitArray)
+ {
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(TypeRelationId, typeOid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for type \"%s\"",
format_type_be(typeOid));
+ }
InvokeObjectPostAlterHook(TypeRelationId, typeOid, 0);
@@ -4571,6 +4615,7 @@ AlterTypeRecurse(Oid typeOid, bool isImplicitArray,
SysScanDesc scan;
ScanKeyData key[1];
HeapTuple domainTup;
+ Form_pg_type typeForm;
/* Since this function recurses, it could be driven to stack overflow */
check_stack_depth();
@@ -4619,6 +4664,45 @@ AlterTypeRecurse(Oid typeOid, bool isImplicitArray,
newtup = heap_modify_tuple(tup, RelationGetDescr(catalog),
values, nulls, replaces);
+ /* Lock dependent objects */
+ typeForm = (Form_pg_type) GETSTRUCT(newtup);
+
+ if (OidIsValid(typeForm->typnamespace))
+ LockNotPinnedObject(NamespaceRelationId, typeForm->typnamespace);
+
+ if (OidIsValid(typeForm->typinput))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typinput);
+
+ if (OidIsValid(typeForm->typoutput))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typoutput);
+
+ if (OidIsValid(typeForm->typreceive))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typreceive);
+
+ if (OidIsValid(typeForm->typsend))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typsend);
+
+ if (OidIsValid(typeForm->typmodin))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typmodin);
+
+ if (OidIsValid(typeForm->typmodout))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typmodout);
+
+ if (OidIsValid(typeForm->typanalyze))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typanalyze);
+
+ if (OidIsValid(typeForm->typsubscript))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typsubscript);
+
+ if (OidIsValid(typeForm->typbasetype))
+ LockNotPinnedObject(TypeRelationId, typeForm->typbasetype);
+
+ if (OidIsValid(typeForm->typcollation))
+ LockNotPinnedObject(CollationRelationId, typeForm->typcollation);
+
+ if (OidIsValid(typeForm->typelem))
+ LockNotPinnedObject(TypeRelationId, typeForm->typelem);
+
CatalogTupleUpdate(catalog, &newtup->t_self, newtup);
/* Rebuild dependencies for this type */
diff --git a/src/backend/rewrite/rewriteDefine.c b/src/backend/rewrite/rewriteDefine.c
index 6cc9a8d8bf..c930eca262 100644
--- a/src/backend/rewrite/rewriteDefine.c
+++ b/src/backend/rewrite/rewriteDefine.c
@@ -155,6 +155,7 @@ InsertRule(const char *rulname,
referenced.objectId = eventrel_oid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, eventrel_oid);
recordDependencyOn(&myself, &referenced,
(evtype == CMD_SELECT) ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/utils/errcodes.txt b/src/backend/utils/errcodes.txt
index b43a24d4bc..ae41f3613c 100644
--- a/src/backend/utils/errcodes.txt
+++ b/src/backend/utils/errcodes.txt
@@ -271,6 +271,7 @@ Section: Class 28 - Invalid Authorization Specification
Section: Class 2B - Dependent Privilege Descriptors Still Exist
2B000 E ERRCODE_DEPENDENT_PRIVILEGE_DESCRIPTORS_STILL_EXIST dependent_privilege_descriptors_still_exist
+2BP02 E ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST dependent_objects_does_not_exist
2BP01 E ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST dependent_objects_still_exist
Section: Class 2D - Invalid Transaction Termination
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6908ca7180..0546bcbe16 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -101,6 +101,8 @@ typedef struct ObjectAddresses ObjectAddresses;
/* in dependency.c */
extern void AcquireDeletionLock(const ObjectAddress *object, int flags);
+extern void LockNotPinnedObjectById(const ObjectAddress *object);
+extern void LockNotPinnedObject(Oid classid, Oid objid);
extern void ReleaseDeletionLock(const ObjectAddress *object);
@@ -172,6 +174,7 @@ extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
+extern bool isObjectPinned(const ObjectAddress *object);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/objectaddress.h b/src/include/catalog/objectaddress.h
index 3a70d80e32..56f746264b 100644
--- a/src/include/catalog/objectaddress.h
+++ b/src/include/catalog/objectaddress.h
@@ -53,6 +53,7 @@ extern void check_object_ownership(Oid roleid,
Node *object, Relation relation);
extern Oid get_object_namespace(const ObjectAddress *address);
+extern bool ObjectByIdExist(const ObjectAddress *address);
extern bool is_objectclass_supported(Oid class_id);
extern const char *get_object_class_descr(Oid class_id);
diff --git a/src/test/isolation/expected/test_dependencies_locks.out b/src/test/isolation/expected/test_dependencies_locks.out
new file mode 100644
index 0000000000..9b645d7aa5
--- /dev/null
+++ b/src/test/isolation/expected/test_dependencies_locks.out
@@ -0,0 +1,129 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_begin s1_create_function_in_schema s2_drop_schema s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_schema: DROP SCHEMA testschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_schema: <... completed>
+ERROR: cannot drop schema testschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_schema s1_create_function_in_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_schema: DROP SCHEMA testschema;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_in_schema: <... completed>
+ERROR: schema testschema does not exist
+
+starting permutation: s1_begin s1_alter_function_schema s2_drop_alterschema s1_commit
+step s1_begin: BEGIN;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema;
+step s2_drop_alterschema: DROP SCHEMA alterschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_alterschema: <... completed>
+ERROR: cannot drop schema alterschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_alterschema s1_alter_function_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_alterschema: DROP SCHEMA alterschema;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema; <waiting ...>
+step s2_commit: COMMIT;
+step s1_alter_function_schema: <... completed>
+ERROR: schema alterschema does not exist
+
+starting permutation: s1_begin s1_create_function_with_argtype s2_drop_foo_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_foo_type: DROP TYPE public.foo; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_type: <... completed>
+ERROR: cannot drop type foo because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_type s1_create_function_with_argtype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_type: DROP TYPE public.foo;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_argtype: <... completed>
+ERROR: type foo does not exist
+
+starting permutation: s1_begin s1_create_function_with_rettype s2_drop_foo_rettype s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1;
+step s2_drop_foo_rettype: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_rettype: <... completed>
+ERROR: cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_rettype s1_create_function_with_rettype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_rettype: DROP DOMAIN id;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_rettype: <... completed>
+ERROR: type id does not exist
+
+starting permutation: s1_begin s1_create_function_with_function s2_drop_function_f s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1;
+step s2_drop_function_f: DROP FUNCTION f(); <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_function_f: <... completed>
+ERROR: cannot drop function f() because other objects depend on it
+
+starting permutation: s2_begin s2_drop_function_f s1_create_function_with_function s2_commit
+step s2_begin: BEGIN;
+step s2_drop_function_f: DROP FUNCTION f();
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_function: <... completed>
+ERROR: function f() does not exist
+
+starting permutation: s1_begin s1_create_domain_with_domain s2_drop_domain_id s1_commit
+step s1_begin: BEGIN;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id;
+step s2_drop_domain_id: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_domain_id: <... completed>
+ERROR: cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_domain_id s1_create_domain_with_domain s2_commit
+step s2_begin: BEGIN;
+step s2_drop_domain_id: DROP DOMAIN id;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_domain_with_domain: <... completed>
+ERROR: type id does not exist
+
+starting permutation: s1_begin s1_create_table_with_type s2_drop_footab_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab);
+step s2_drop_footab_type: DROP TYPE public.footab; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_footab_type: <... completed>
+ERROR: cannot drop type footab because other objects depend on it
+
+starting permutation: s2_begin s2_drop_footab_type s1_create_table_with_type s2_commit
+step s2_begin: BEGIN;
+step s2_drop_footab_type: DROP TYPE public.footab;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab); <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_table_with_type: <... completed>
+ERROR: type footab does not exist
+
+starting permutation: s1_begin s1_create_server_with_fdw_wrapper s2_drop_fdw_wrapper s1_commit
+step s1_begin: BEGIN;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_fdw_wrapper: <... completed>
+ERROR: cannot drop foreign-data wrapper fdw_wrapper because other objects depend on it
+
+starting permutation: s2_begin s2_drop_fdw_wrapper s1_create_server_with_fdw_wrapper s2_commit
+step s2_begin: BEGIN;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_server_with_fdw_wrapper: <... completed>
+ERROR: foreign-data wrapper fdw_wrapper does not exist
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 6da98cffac..ef6a7075bc 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -117,3 +117,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: test_dependencies_locks
diff --git a/src/test/isolation/specs/test_dependencies_locks.spec b/src/test/isolation/specs/test_dependencies_locks.spec
new file mode 100644
index 0000000000..5d04dfe9dc
--- /dev/null
+++ b/src/test/isolation/specs/test_dependencies_locks.spec
@@ -0,0 +1,89 @@
+setup
+{
+ CREATE SCHEMA testschema;
+ CREATE SCHEMA alterschema;
+ CREATE TYPE public.foo as enum ('one', 'two');
+ CREATE TYPE public.footab as enum ('three', 'four');
+ CREATE DOMAIN id AS int;
+ CREATE FUNCTION f() RETURNS int LANGUAGE SQL RETURN 1;
+ CREATE FUNCTION public.falter() RETURNS int LANGUAGE SQL RETURN 1;
+ CREATE FOREIGN DATA WRAPPER fdw_wrapper;
+}
+
+teardown
+{
+ DROP FUNCTION IF EXISTS testschema.foo();
+ DROP FUNCTION IF EXISTS fooargtype(num foo);
+ DROP FUNCTION IF EXISTS footrettype();
+ DROP FUNCTION IF EXISTS foofunc();
+ DROP FUNCTION IF EXISTS public.falter();
+ DROP FUNCTION IF EXISTS alterschema.falter();
+ DROP DOMAIN IF EXISTS idid;
+ DROP SERVER IF EXISTS srv_fdw_wrapper;
+ DROP TABLE IF EXISTS tabtype;
+ DROP SCHEMA IF EXISTS testschema;
+ DROP SCHEMA IF EXISTS alterschema;
+ DROP TYPE IF EXISTS public.foo;
+ DROP TYPE IF EXISTS public.footab;
+ DROP DOMAIN IF EXISTS id;
+ DROP FUNCTION IF EXISTS f();
+ DROP FOREIGN DATA WRAPPER IF EXISTS fdw_wrapper;
+}
+
+session "s1"
+
+step "s1_begin" { BEGIN; }
+step "s1_create_function_in_schema" { CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_argtype" { CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_rettype" { CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; }
+step "s1_create_function_with_function" { CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; }
+step "s1_alter_function_schema" { ALTER FUNCTION public.falter() SET SCHEMA alterschema; }
+step "s1_create_domain_with_domain" { CREATE DOMAIN idid as id; }
+step "s1_create_table_with_type" { CREATE TABLE tabtype(a footab); }
+step "s1_create_server_with_fdw_wrapper" { CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; }
+step "s1_commit" { COMMIT; }
+
+session "s2"
+
+step "s2_begin" { BEGIN; }
+step "s2_drop_schema" { DROP SCHEMA testschema; }
+step "s2_drop_alterschema" { DROP SCHEMA alterschema; }
+step "s2_drop_foo_type" { DROP TYPE public.foo; }
+step "s2_drop_foo_rettype" { DROP DOMAIN id; }
+step "s2_drop_footab_type" { DROP TYPE public.footab; }
+step "s2_drop_function_f" { DROP FUNCTION f(); }
+step "s2_drop_domain_id" { DROP DOMAIN id; }
+step "s2_drop_fdw_wrapper" { DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; }
+step "s2_commit" { COMMIT; }
+
+# function - schema
+permutation "s1_begin" "s1_create_function_in_schema" "s2_drop_schema" "s1_commit"
+permutation "s2_begin" "s2_drop_schema" "s1_create_function_in_schema" "s2_commit"
+
+# alter function - schema
+permutation "s1_begin" "s1_alter_function_schema" "s2_drop_alterschema" "s1_commit"
+permutation "s2_begin" "s2_drop_alterschema" "s1_alter_function_schema" "s2_commit"
+
+# function - argtype
+permutation "s1_begin" "s1_create_function_with_argtype" "s2_drop_foo_type" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_type" "s1_create_function_with_argtype" "s2_commit"
+
+# function - rettype
+permutation "s1_begin" "s1_create_function_with_rettype" "s2_drop_foo_rettype" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_rettype" "s1_create_function_with_rettype" "s2_commit"
+
+# function - function
+permutation "s1_begin" "s1_create_function_with_function" "s2_drop_function_f" "s1_commit"
+permutation "s2_begin" "s2_drop_function_f" "s1_create_function_with_function" "s2_commit"
+
+# domain - domain
+permutation "s1_begin" "s1_create_domain_with_domain" "s2_drop_domain_id" "s1_commit"
+permutation "s2_begin" "s2_drop_domain_id" "s1_create_domain_with_domain" "s2_commit"
+
+# table - type
+permutation "s1_begin" "s1_create_table_with_type" "s2_drop_footab_type" "s1_commit"
+permutation "s2_begin" "s2_drop_footab_type" "s1_create_table_with_type" "s2_commit"
+
+# server - foreign data wrapper
+permutation "s1_begin" "s1_create_server_with_fdw_wrapper" "s2_drop_fdw_wrapper" "s1_commit"
+permutation "s2_begin" "s2_drop_fdw_wrapper" "s1_create_server_with_fdw_wrapper" "s2_commit"
diff --git a/src/test/modules/test_oat_hooks/expected/alter_table.out b/src/test/modules/test_oat_hooks/expected/alter_table.out
index 8cbacca2c9..df8d276dfc 100644
--- a/src/test/modules/test_oat_hooks/expected/alter_table.out
+++ b/src/test/modules/test_oat_hooks/expected/alter_table.out
@@ -37,6 +37,8 @@ NOTICE: in object access: superuser attempting create (subId=0x0) [internal]
NOTICE: in object access: superuser finished create (subId=0x0) [internal]
NOTICE: in object access: superuser attempting create (subId=0x0) [internal]
NOTICE: in object access: superuser finished create (subId=0x0) [internal]
+NOTICE: in object access: superuser attempting namespace search (subId=0x0) [no report on violation, allowed]
+NOTICE: in object access: superuser finished namespace search (subId=0x0) [no report on violation, allowed]
NOTICE: in process utility: superuser finished CREATE TABLE
CREATE RULE test_oat_notify AS
ON UPDATE TO test_oat_schema.test_oat_tab
@@ -62,8 +64,6 @@ BEGIN
END IF;
END; $$;
NOTICE: in process utility: superuser attempting CREATE FUNCTION
-NOTICE: in object access: superuser attempting namespace search (subId=0x0) [no report on violation, allowed]
-NOTICE: in object access: superuser finished namespace search (subId=0x0) [no report on violation, allowed]
NOTICE: in object access: superuser attempting create (subId=0x0) [explicit]
NOTICE: in object access: superuser finished create (subId=0x0) [explicit]
NOTICE: in process utility: superuser finished CREATE FUNCTION
diff --git a/src/test/modules/test_oat_hooks/expected/test_oat_hooks.out b/src/test/modules/test_oat_hooks/expected/test_oat_hooks.out
index effdc49145..da6d931994 100644
--- a/src/test/modules/test_oat_hooks/expected/test_oat_hooks.out
+++ b/src/test/modules/test_oat_hooks/expected/test_oat_hooks.out
@@ -86,6 +86,8 @@ NOTICE: in object access: superuser attempting create (subId=0x0) [internal]
NOTICE: in object access: superuser finished create (subId=0x0) [internal]
NOTICE: in object access: superuser attempting create (subId=0x0) [internal]
NOTICE: in object access: superuser finished create (subId=0x0) [internal]
+NOTICE: in object access: superuser attempting namespace search (subId=0x0) [no report on violation, allowed]
+NOTICE: in object access: superuser finished namespace search (subId=0x0) [no report on violation, allowed]
NOTICE: in process utility: superuser finished CREATE TABLE
CREATE INDEX regress_test_table_t_idx ON regress_test_table (t);
NOTICE: in process utility: superuser attempting CREATE INDEX
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 79cf82b5ae..1ed23c54cb 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2867,11 +2867,12 @@ begin;
alter table alterlock2
add constraint alterlock2nv foreign key (f1) references alterlock (f1) NOT VALID;
select * from my_locks order by 1;
- relname | max_lockmode
-------------+-----------------------
- alterlock | ShareRowExclusiveLock
- alterlock2 | ShareRowExclusiveLock
-(2 rows)
+ relname | max_lockmode
+----------------+-----------------------
+ alterlock | ShareRowExclusiveLock
+ alterlock2 | ShareRowExclusiveLock
+ alterlock_pkey | AccessShareLock
+(3 rows)
commit;
begin;
--
2.34.1
--hYoucLb4AamrrTZZ--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 331+ messages in thread
* Add per-backend AIO statistics
@ 2026-07-07 11:02 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:00 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
0 siblings, 2 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-07-07 11:02 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
Hi hackers,
Currently to monitor AIO we can use:
1/ pg_aios that lists all AIO handles that are currently in use. That shows
what's happening right now, but not what has happened.
2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
was done. There's no way to see whether IOs ran synchronously or
asynchronously, whether a backend was stalling on handle exhaustion, or how
completions are distributed across backends.
This patch helps answering those questions by exposing cumulative per-backend
AIO counters:
- started: total AIO operations initiated
- executed_sync: IOs executed synchronously (fallback path)
- executed_async: IOs submitted asynchronously
- completed_self: IO completions processed by the issuing backend
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times waited for a free AIO handle
- submitted: number of submit calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion patterns.
That helps see how IO completion work is distributed and could help interpret
per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
As far as the technical implementation:
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns one row based on the PID provided in input.
pgstat_flush_backend() gains a new flag value, able to control the flush of the
AIO stats.
This patch relies mostly on the infrastructure provided by 9aea73fc61d4, that
has introduced backend statistics.
The overhead (4 functions calls and counters increments) kind of follow the same
patterns as pgstat_count_backend_io_op() and I did not observe measurable
regression (I did not expect to). Also that does not add that much memory
per-backend: PgStat_AioCounters is 56 bytes.
There is no "double" counting as a global view to show those counters does not
exist. I think that's better to start with the per-backend side of it and see
if we want to also add a global view. For example, completed_other identifies
which backends did IOs for other backends. Also this allows correlating with
pg_stat_activity and pg_stat_get_backend_io().
Examples based on Franck's blog post [1]:
1/ query the smalldocs table:
postgres=# select count(*),avg(length(data)) from smalldocs;
count | avg
---------+-----------------------
1024000 | 1024.0000000000000000
(1 row)
postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
3125 | 46 | 3079 | 46 | 0 | 0 | 3078 | 2026-07-07 09:28:27.412136+00
We can see that the sequential scan fully benefits from AIO.
2/ query the largedocs table:
postgres=# select count(*),avg(length(data)) from largedocs;
count | avg
-------+----------------------
1000 | 1048576.000000000000
(1 row)
postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
121154 | 121150 | 4 | 121150 | 0 | 0 | 4 | 2026-07-07 09:35:00.504872+00
We can see that the sequential scan bypasses AIO.
Looking forward to your feedback.
[1]: https://dev.to/franckpachot/iouring-buffered-reads-in-postgresql-19-iouring-mcn
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
@ 2026-07-08 06:00 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
1 sibling, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-07-08 06:00 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
Hi,
On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> postgres=# select count(*),avg(length(data)) from smalldocs;
> count | avg
> ---------+-----------------------
> 1024000 | 1024.0000000000000000
> (1 row)
>
> postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
> started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
> ---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
> 3125 | 46 | 3079 | 46 | 0 | 0 | 3078 | 2026-07-07 09:28:27.412136+00
>
> We can see that the sequential scan fully benefits from AIO.
>
> 2/ query the largedocs table:
>
> postgres=# select count(*),avg(length(data)) from largedocs;
> count | avg
> -------+----------------------
> 1000 | 1048576.000000000000
> (1 row)
>
> postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
> started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
> ---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
> 121154 | 121150 | 4 | 121150 | 0 | 0 | 4 | 2026-07-07 09:35:00.504872+00
>
> We can see that the sequential scan bypasses AIO.
I was just doing some AIO experiments and was using the new pg_stat_get_backend_aio()
function.
So, while at it, sharing more examples here:
3/ pg_stat_get_backend_aio() and pg_stat_get_backend_io() correlation
postgres=# SELECT executed_sync, executed_async FROM pg_stat_get_backend_aio(pg_backend_pid());
executed_sync | executed_async
---------------+----------------
46 | 3088
(1 row)
postgres=# SELECT object, context, reads, read_bytes FROM pg_stat_get_backend_io(pg_backend_pid());
object | context | reads | read_bytes
---------------+-----------+-------+------------
relation | bulkread | 3088 | 401580032
relation | bulkwrite | 0 | 0
relation | init | 0 | 0
relation | normal | 46 | 376832
relation | vacuum | 0 | 0
temp relation | normal | 0 | 0
wal | init | |
wal | normal | 0 | 0
(8 rows)
We can see that the "executed_sync" matches the reads "normal" context and that
the "executed_async" matches the reads "bulkread" context.
4/ io_uring and multiple backends
postgres=# SELECT a.pid,
(pg_stat_get_backend_aio(a.pid)).completed_other
FROM pg_stat_activity a
WHERE a.backend_type = 'client backend';
pid | completed_other
---------+-----------------
1911889 | 245
1911892 | 511
1911912 | 147
1911933 | 161
(4 rows)
We can see that the backends completed AIO on behalf of other backends, which
makes fully sense in io_uring mode.
5/ io_max_concurrency = 4
postgres=# SELECT started, handle_waits FROM pg_stat_get_backend_aio(pg_backend_pid());
started | handle_waits
---------+--------------
3139 | 3026
(1 row)
We can see that the backend had to wait for free AIO handles on 96% of its IOs.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
@ 2026-07-08 06:52 ` Michael Paquier <michael@paquier.xyz>
2026-07-08 08:15 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 331+ messages in thread
From: Michael Paquier @ 2026-07-08 06:52 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org; Andres Freund <andres@anarazel.de>
On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> what's happening right now, but not what has happened.
>
> 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> was done. There's no way to see whether IOs ran synchronously or
> asynchronously, whether a backend was stalling on handle exhaustion, or how
> completions are distributed across backends.
While the information may be useful, one thing that sounds very
important to me is how this impacts workloads by default.
Andres is usually able to catch bottlenecks that everybody else is
unable to see, so perhaps checking with him the location of these
extra function calls would be a good first step. Your proposal goes
down to pgaio_io_stage(), pgaio_io_process_completion() and
pgaio_submit_staged() to track these counter increments.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../ak3zpA2HHh5CxQK8@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
@ 2026-07-08 08:15 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
1 sibling, 0 replies; 331+ messages in thread
From: Bertrand Drouvot @ 2026-07-08 08:15 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-hackers@lists.postgresql.org; Andres Freund <andres@anarazel.de>
Hi,
On Wed, Jul 08, 2026 at 03:52:20PM +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful,
Thanks for looking at it!
> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step. Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.
yeah, and also to 1/ confirm that I did understand this area of the AIO code
correctly and 2/ see if other counters could make sense.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
@ 2026-07-08 18:08 ` Andres Freund <andres@anarazel.de>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
1 sibling, 1 reply; 331+ messages in thread
From: Andres Freund @ 2026-07-08 18:08 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; pgsql-hackers@lists.postgresql.org
Hi,
On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful, one thing that sounds very
> important to me is how this impacts workloads by default.
> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step. Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.
I think the overhead might be ok, but I am rather doubtful that all of this
information is actually useful. You're adding quite a few counters for each
IO, do we actually need that?
E.g. what do we gain from counting:
- started (if you want to see the number of IOs that are in progress,
cumulative stats are the wrong tool)
- executed_async (that's just the number of IOs minus executed_sync)
- completed_self (that's just the number of IOs minus executed_other)
Separately, I'm doubtful it makes sense to have only per-backend stats for
this. I think you'd almost always want the stats for exited backend
(e.g. parallel workers) too.
Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
architectural direction. It'll just end up implementing all kinds of stats,
since we'll incrementally want more and more per-backend stats. I think what
we'd want is rather something where for each applicable stats kind we have a
shared counter for all exited backends and then per-backend counters for live
backends, with helpers to aggregate the exited + live stats to a total.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <andres@anarazel.de>
@ 2026-07-09 04:19 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-10 04:56 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 1 reply; 331+ messages in thread
From: Bertrand Drouvot @ 2026-07-09 04:19 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-hackers@lists.postgresql.org
Hi,
On Wed, Jul 08, 2026 at 02:08:00PM -0400, Andres Freund wrote:
> Hi,
>
> On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> > On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > > what's happening right now, but not what has happened.
> > >
> > > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > > was done. There's no way to see whether IOs ran synchronously or
> > > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > > completions are distributed across backends.
> >
> > While the information may be useful, one thing that sounds very
> > important to me is how this impacts workloads by default.
>
>
> > Andres is usually able to catch bottlenecks that everybody else is
> > unable to see, so perhaps checking with him the location of these
> > extra function calls would be a good first step. Your proposal goes
> > down to pgaio_io_stage(), pgaio_io_process_completion() and
> > pgaio_submit_staged() to track these counter increments.
>
> I think the overhead might be ok,
Thanks for the feedback.
> but I am rather doubtful that all of this
> information is actually useful. You're adding quite a few counters for each
> IO, do we actually need that?
>
> E.g. what do we gain from counting:
> - started (if you want to see the number of IOs that are in progress,
> cumulative stats are the wrong tool)
> - executed_async (that's just the number of IOs minus executed_sync)
> - completed_self (that's just the number of IOs minus executed_other)
Yeah, we can remove some fields (as they're derivable).
> Separately, I'm doubtful it makes sense to have only per-backend stats for
> this. I think you'd almost always want the stats for exited backend
> (e.g. parallel workers) too.
Indeed, adding a global view would capture their activity.
> Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
> architectural direction. It'll just end up implementing all kinds of stats,
> since we'll incrementally want more and more per-backend stats. I think what
> we'd want is rather something where for each applicable stats kind we have a
> shared counter for all exited backends and then per-backend counters for live
> backends, with helpers to aggregate the exited + live stats to a total.
That's a very nice proposal that would avoid the double counting. OTOH, that's
also a major re-design that would benefit all existing per-backend stats kinds.
I can see 2 options:
1/
step 1: Implement per-backend AIO stats (like proposed taking into account your
remark about useless, derivable fields) + a global view.
step 2: work on the re-design
2/
step 1: work on the redesign
step 2: Add AIO stats based on the re-design
The pros of 1/ is that step 1 would most probably land in 20, providing more user
visibility (+ it could be used or improved during the AIO write project). Step 2
is a much larger project that might not land in 20.
The cons, would be double counting (as there is no need to try to implement
something like [1] as we are going to re-design anyway).
I'll be tempted to vote for 1/ to provide faster added value. What do you (Andres,
Michael) think?
[1]: https://postgr.es/m/aNVWe2tR1jj5Tsct@ip-10-97-1-34.eu-west-3.compute.internal
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <andres@anarazel.de>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
@ 2026-07-10 04:56 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-30 11:44 ` Re: Add per-backend AIO statistics solai v <solai.cdac@gmail.com>
0 siblings, 1 reply; 331+ messages in thread
From: Bertrand Drouvot @ 2026-07-10 04:56 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-hackers@lists.postgresql.org
Hi,
On Thu, Jul 09, 2026 at 04:19:26AM +0000, Bertrand Drouvot wrote:
> Hi,
>
> On Wed, Jul 08, 2026 at 02:08:00PM -0400, Andres Freund wrote:
>
> > Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
> > architectural direction. It'll just end up implementing all kinds of stats,
> > since we'll incrementally want more and more per-backend stats. I think what
> > we'd want is rather something where for each applicable stats kind we have a
> > shared counter for all exited backends and then per-backend counters for live
> > backends, with helpers to aggregate the exited + live stats to a total.
>
> That's a very nice proposal that would avoid the double counting. OTOH, that's
> also a major re-design that would benefit all existing per-backend stats kinds.
>
> I can see 2 options:
>
> 1/
>
> step 1: Implement per-backend AIO stats (like proposed taking into account your
> remark about useless, derivable fields) + a global view.
> step 2: work on the re-design
>
> 2/
>
> step 1: work on the redesign
> step 2: Add AIO stats based on the re-design
>
> The pros of 1/ is that step 1 would most probably land in 20, providing more user
> visibility (+ it could be used or improved during the AIO write project). Step 2
> is a much larger project that might not land in 20.
>
> The cons, would be double counting (as there is no need to try to implement
> something like [1] as we are going to re-design anyway).
>
> I'll be tempted to vote for 1/ to provide faster added value. What do you (Andres,
> Michael) think?
Actually, there is no rush to merge the per-backend AIO stats (we still have
plenty of time for 20). So let's try option 2 and implement the new design first
and see where it goes. I'll create a dedicated thread once ready.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 331+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <andres@anarazel.de>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-10 04:56 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
@ 2026-07-30 11:44 ` solai v <solai.cdac@gmail.com>
0 siblings, 0 replies; 331+ messages in thread
From: solai v @ 2026-07-30 11:44 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Michael Paquier <michael@paquier.xyz>; pgsql-hackers@lists.postgresql.org
Hi,
I tested the patch "v1-0001-Add-per-backend-AIO-statistics.patch".
The patch applied cleanly without conflicts. I rebuilt PostgreSQL,
installed the patched binaries, initialized a fresh cluster using
initdb, and verified that the new function pg_stat_get_backend_aio()
was successfully added and available.
For functional testing, I first queried the new function before
generating any workload:
started = 74
executed_sync = 73
executed_async = 1
completed_self = 73
completed_other = 0
handle_waits = 0
submitted = 1
To generate AIO activity, I created a table with 1,000,000 rows, ran
ANALYZE, and executed SELECT COUNT(*) on the table. After the
workload, I queried pg_stat_get_backend_aio(pg_backend_pid()) again
and observed:
started = 26632
executed_sync = 137
executed_async = 26495
completed_self = 224
completed_other = 0
handle_waits = 0
submitted = 26480
The results show a significant increase in the started,
executed_async, and submitted counters after the workload, which is
consistent with the expected increase in asynchronous I/O activity.
The completed_other and handle_waits counters remained at zero during
my testing, which was expected for this workload.
Overall, the new function returned the expected per-backend AIO
statistics, and the observed values reflected the generated workload.
I did not encounter any build or functional issues while testing the
patch.
Regards
Solai
^ permalink raw reply [nested|flat] 331+ messages in thread
end of thread, other threads:[~2026-07-30 11:44 UTC | newest]
Thread overview: 331+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-03-29 15:43 [PATCH v16] Avoid orphaned objects dependencies Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:00 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <michael@paquier.xyz>
2026-07-08 08:15 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <andres@anarazel.de>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-10 04:56 ` Re: Add per-backend AIO statistics Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-07-30 11:44 ` Re: Add per-backend AIO statistics solai v <solai.cdac@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox